feature: add dfdaemon.Server

Signed-off-by: inoc603<EMAIL_ADDRESS>
Ⅰ. Describe what this PR did
Add dfdaemon.Server and use the same port for transparent proxy and reverse proxy.
Ⅱ. Does this pull request fix one issue?

fixes #534
Ⅲ. Why don't you add test cases (unit test/integration test)? (你真的觉得不需要加测试吗？)
Ⅳ. Describe how to verify it
Use 65001 instead of 65002 for the transparent proxy.
Ⅴ. Special notes for reviews
I haven't change the registries configuration as suggested in #534. This is currently a proof of concept to demonstrate that the idea will work.
I didn't change the use of global variables in the dfdaemon package, but I think it's necessary to remove them to make reasoning about the code and writing tests easier. Maybe I'll open another PR for it.

Codecov Report

Merging #536 into master will decrease coverage by 0.21%.
The diff coverage is 11.76%.

@@            Coverage Diff             @@
##           master     #536      +/-   ##
==========================================
- Coverage   52.18%   51.96%   -0.22%
==========================================
  Files          83       82       -1
  Lines        4461     4468       +7
==========================================
- Hits         2328     2322       -6
- Misses       1980     1994      +14
+ Partials      153      152       -1

Impacted Files
Coverage Δ

dfdaemon/proxy/proxy.go
10.82% <11.76%> (-0.44%)
:arrow_down:

supernode/daemon/mgr/scheduler/manager.go
25.6% <0%> (+0.8%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2fca259...cdbb2c2. Read the comment docs.

@lowzj any thoughts on the registries field I mentioned before?

@inoc603 yeah, registry_mirror and proxies are much clearer and better than before, we're very looking forward to if you want to keep working on this.

@lowzj the registry cli parameter currently has no effect, should we remove it or use it as a short hand for registries.remote ?

@inoc603 It's better to remain registry option, and it has high priority than config.

@inoc603 It's better to remain registry option, and it has higher priority than config.

Sure. In order to do this, I changed the default value for the flag to an empty string. PTAL

@inoc603 It's better to remain registry option, and it has higher priority than config.

Sure. In order to do this, I changed the default value for the flag to an empty string. PTAL

@inoc603  The registry option and registry_mirror config are ineffective now, maybe somewhere is wrong.

@inoc603 The registry option and registry_mirror config are ineffective now, maybe somewhere is wrong.

My bad. I forgot to pass the mirror config to proxy. It should be fixed now.

@inoc603 Thanks for your excellent  work!! Could you rebase your code and push again?

@lowzj Rebased.


Make use of the LSM Tigerbeetle release for testcontainers (once ready)

@sentientwaffle said there are no major demons anymore
BUT: does not work on Apple M1 (however, docker works)
TB and node client now in the same repo
update TB together with node client

@koekiebox  remapped error codes that need to be cleaned up in packages/backend/src/accounting/transfers.ts
https://github.com/interledger/rafiki/pull/452/files#diff-e220292cea3c986078ff4fb24d3d3d9f4c949ae28ad3412026743f4e0aec6580

Blocked by

https://github.com/tigerbeetledb/tigerbeetle-node/issues/43

Fix:

https://github.com/tigerbeetledb/tigerbeetle/pull/372

Since https://github.com/tigerbeetledb/tigerbeetle/pull/430, tigerbeetle-node v0.11.9 works on macOS 13, and is able to to be installed in rafiki. However, when trying to start up the local environment, I'm trying to debug this error when start:
2023-01-27 18:08:02 node:internal/modules/cjs/loader:1302
2023-01-27 18:08:02   return process.dlopen(module, path.toNamespacedPath(filename));
2023-01-27 18:08:02                  ^
2023-01-27 18:08:02
2023-01-27 18:08:02 Error: /usr/lib/aarch64-linux-gnu/libc.so: invalid ELF header
2023-01-27 18:08:02     at Module._extensions..node (node:internal/modules/cjs/loader:1302:18)
2023-01-27 18:08:02     at Module.load (node:internal/modules/cjs/loader:1081:32)
2023-01-27 18:08:02     at Module._load (node:internal/modules/cjs/loader:922:12)
2023-01-27 18:08:02     at Module.require (node:internal/modules/cjs/loader:1105:19)
2023-01-27 18:08:02     at require (node:internal/modules/cjs/helpers:103:18)
2023-01-27 18:08:02     at Object.<anonymous> (/workspace/node_modules/.pnpm/tigerbeetle-node@0.11.9/node_modules/tigerbeetle-node/dist/index.js:4:17)
2023-01-27 18:08:02     at Module._compile (node:internal/modules/cjs/loader:1218:14)
2023-01-27 18:08:02     at Module._extensions..js (node:internal/modules/cjs/loader:1272:10)
2023-01-27 18:08:02     at Module.load (node:internal/modules/cjs/loader:1081:32)
2023-01-27 18:08:02     at Module._load (node:internal/modules/cjs/loader:922:12) {
2023-01-27 18:08:02   code: 'ERR_DLOPEN_FAILED'

Currently looking into this

@mkurapov I'll try that now that the fix is in.


Enhance dry-run release tasks

Add —dry-run option support for the changelog and gh-release tasks.
Skip the bundle compilation, unit testing and bundle distribution tasks when —dry-run is specified.

:+1:

Not sure why this has merge conflicts :frowning:

@alice merge conflicts are resolved here, do you want to merge?

@loginx can merge this any time, he has write access. Otherwise, I'm happy to do it tomorrow (assuming he's busy with other stuff)

I just returned from vacation. I'll take care of this today.


~Inverted~ names on silkscreen not supported

If you have text on the silkscreen layer that uses ~ to add a bar on the top, then this isn't supported on BOM html output display.
i.e.
If you have text on the silkscreen layer which contains ~Reset~ then this will appear within Kicad as Reset with a bar over the top. However, within the BOM html output, it appears as ~Reset~
Yeah, ok, this isn't a high priority to fix.

I have a fix ready but it's pending resolution on https://gitlab.com/kicad/code/kicad/-/issues/4396
Text rendering logic in kicad may change slightly and I'll follow suit.


Irradiance probes exceeded

Hi, The work your doing is great and very helpfull to learn from as this progresses.
Ive tried playing with the new irradiance branch, But when ever i run a scene i get an exit with number of probes exceeded. Hope it helps.
J

Closing, since Baikal has been moved into a separate repo


Support Node.js 8

Dev dependency on v8-inspector which in turn depends on v8-debug, which fails to compile on Node.js 8 is preventing us from support Node.js 8.

This is specially important now since Node.js 8.x is about to enter the Active LTS status.

I concur. I think we're going to have to disable our v8-inspector dependency for now.

The node-inspector dependency was the culprit after. Removed and now Node.js 8 is supported in the latest release after #316. Closing.


hassio 2021.1.4 - The reolink_dev platform for the camera integration does not support platform setup. Please remove it from your config.

Describe the bug
reolink_dev worked fine until a few hours ago.
Then I updated to the latest version:

HACS
reolink_dev

I restarted HASSIO, but unfortunately I did not check if the cameras still work or not.
Then I updatet HASSIO from 2021.1.1 to 2021.1.4
And only then I checked the cameras. reolink cameras stopped to work. I get these kind of errors in the home-assitent.log

2021-01-17 10:08:11 ERROR (MainThread) [homeassistant.components.camera] The reolink_dev platform for the camera integration does not support platform setup. Please remove it from your config.
2021-01-17 10:08:11 ERROR (MainThread) [homeassistant.components.camera] The reolink_dev platform for the camera integration does not support platform setup. Please remove it from your config.

To Reproduce
use the latest version of hassio, hacs, reolink_dev
here is an example of my configuration, nothing special:

platform: reolink_dev
host: <IP_ADDRESS>xx
username: yyyyy
password: !secret camera_reolink_password
name: Garage

Expected behavior
I want to be able to use the cameras again
Screenshots
Environment:
Please provide useful information about your environment, like:

Home Assistant version
2021.1.4
Reolink camera model
any reolink camera
Camera firmware number

Additional context
Add any other context about the problem here.

That's right. From now, you can just set it up from the integrations menu. Please see the readme.

That's right. From now, you can just set it up from the integrations menu. Please see the readme.

it looks like it was this small comment in the release?

If you are migrating from <0.7, please remove all existing config and use the config flow to add your camera(s).

It was not clear for me that this was a breaking change.
I would suggest marking this in the release history as "breaking change"

it looks like it was this small comment in the release?

If you are migrating from <0.7, please remove all existing config and use the config flow to add your camera(s).

It was not clear for me that this was a breaking change.
I would suggest marking this in the release history as "breaking change"

You're right! I'll try to change this comment.

 I'll try to change this comment.


Denon AVR : No mapping for network audio source BT

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.38.3
Python release (python3 --version):
3.4.2
Component/platform:
media_player:
  - platform: denonavr

Description of problem:
Every 10 seconds I've got a log this type;
17-02-15 07:13:02 ERROR (Thread-12) [DenonAVR] No mapping for network audio source BT
and my Denon AVR- X1300W is shut off..
Same log if my Denon AVR- X1300W is on, and even with a bluetooth device connected..
Expected:
No logs at all
Problem-relevant configuration.yaml entries and steps to reproduce:

Just put this line in my configuration.yaml:

media_player:
  - platform: denonavr
    host: <IP_ADDRESS>

Traceback (if applicable):

Additional info:
17-02-15 07:13:02 ERROR (Thread-12) [DenonAVR] No mapping for network audio source BT
17-02-15 07:13:44 ERROR (Thread-3) [DenonAVR] No mapping for network audio source BT
17-02-15 07:13:55 ERROR (Thread-10) [DenonAVR] No mapping for network audio source BT
17-02-15 07:14:06 ERROR (Thread-7) [DenonAVR] No mapping for network audio source BT
17-02-15 07:14:16 ERROR (Thread-4) [DenonAVR] No mapping for network audio source BT
17-02-15 07:14:27 ERROR (Thread-5) [DenonAVR] No mapping for network audio source BT
17-02-15 07:14:38 ERROR (Thread-7) [DenonAVR] No mapping for network audio source BT
17-02-15 07:14:49 ERROR (Thread-8) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:00 ERROR (Thread-11) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:11 ERROR (Thread-5) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:22 ERROR (Thread-11) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:33 ERROR (Thread-3) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:44 ERROR (Thread-8) [DenonAVR] No mapping for network audio source BT
17-02-15 07:15:55 ERROR (Thread-6) [DenonAVR] No mapping for network audio source BT
17-02-15 07:16:06 ERROR (Thread-3) [DenonAVR] No mapping for network audio source BT
17-02-15 07:16:17 ERROR (Thread-11) [DenonAVR] No mapping for network audio source BT
17-02-15 07:16:28 ERROR (Thread-3) [DenonAVR] No mapping for network audio source BT
17-02-15 07:16:39 ERROR (Thread-4) [DenonAVR] No mapping for network audio source BT
17-02-15 07:16:50 ERROR (Thread-3) [DenonAVR] No mapping for network audio source BT
17-02-15 07:17:01 ERROR (Thread-4) [DenonAVR] No mapping for network audio source BT
17-02-15 07:17:12 ERROR (Thread-6) [DenonAVR] No mapping for network audio source BT
17-02-15 07:17:23 ERROR (Thread-4) [DenonAVR] No mapping for network audio source BT
17-02-15 07:17:34 ERROR (Thread-5) [DenonAVR] No mapping for network audio source BT

The same error with Denon AVR-X2100W when input switched to Bluetooth and there is no Bluetooth on the sources list in HA.

Same issue with Input change to Network players:
[code]
17-02-19 17:35:15 ERROR (Thread-8) [DenonAVR] No mapping for network audio source NET
17-02-19 17:35:26 ERROR (Thread-6) [DenonAVR] No mapping for network audio source NET
17-02-19 17:35:37 ERROR (Thread-10) [DenonAVR] No mapping for network audio source NET
17-02-19 17:35:48 ERROR (Thread-5) [DenonAVR] No mapping for network audio source NET
[/code]

The error seems to have disappeared on 0.38.4
Can others confirm it ?

No, in my HA 0.38.4 errors still exist in log.

I found a workaround, not perfect I admit:
Add this in your configuration.yaml file:
logger:
  default: error
  logs:
    DenonAVR: fatal

You'll miss all others "error" log, but I didn't have any yet.

I also get these errors also:
17-03-22 11:27:24 ERROR (Thread-11) [DenonAVR] No mapping for network audio source NET
17-03-22 11:27:35 ERROR (Thread-6) [DenonAVR] No mapping for network audio source NET
But ONLY when the receiver is turned of after playing Spotify. When I change input to CBL/SAT before turning off the receiver I get no errors in the log...
Maybe this is useful for someone how can fix this in Home Assistant?

Just upgraded to 0.45.1 and getting similar error. Mine is for audio source in general. Whenever I change sources it errors on the selected source. Also can no longer change sources of receiver from within HA. Besides the workaround is there any progress on this?  My receiver is the AVR-2312CI. If it matters I have edited the names that are displayed from original generic names Denon provided.
Prior to upgrading had no issues with exception of contacting receiver when powered off until it set network access option to remain active when powered off.

Seems to be fixed in 0.46.0?

Still getting a similar issue with 0.48.0. When it was on Media Player I get this continuously appearing in the logs:
2017-07-02 10:57:59 ERROR (Thread-6) [DenonAVR] No mapping for source MPLAY

After update HA to 0.48 one of the sources (HDMI input) generate error every 10 seconds.
2017-07-05 10:57:27 ERROR (Thread-6) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:57:38 ERROR (Thread-3) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:57:49 ERROR (Thread-9) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:58:00 ERROR (Thread-10) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:58:11 ERROR (Thread-4) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:58:21 ERROR (Thread-6) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:58:22 ERROR (Thread-10) [DenonAVR] No mapping for source MPLAY
2017-07-05 10:58:23 ERROR (Thread-10) [DenonAVR] Host denon.local returned malformed XML for: /goform/Deviceinfo.xml
2017-07-05 10:58:23 ERROR (Thread-10) [DenonAVR] Connection error: Receiver sources list empty. Please check if device is powered on.
2017-07-05 10:58:23 ERROR (Thread-10) [DenonAVR] Input function list for Denon receiver at host denon.bieniu.lan could not be updated.

I have the same issue as bieniu:
2017-07-07 09:58:20 ERROR (Thread-9) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:58:31 ERROR (Thread-21) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:58:42 ERROR (Thread-16) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:58:53 ERROR (Thread-22) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:04 ERROR (Thread-14) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:15 ERROR (Thread-8) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:26 ERROR (Thread-10) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:37 ERROR (Thread-5) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:48 ERROR (Thread-20) [DenonAVR] No mapping for source MPLAY 2017-07-07 09:59:59 ERROR (Thread-13) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:00:10 ERROR (Thread-21) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:00:21 ERROR (Thread-15) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:00:32 ERROR (Thread-13) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:00:43 ERROR (Thread-21) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:00:54 ERROR (Thread-12) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:01:05 ERROR (Thread-5) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:01:16 ERROR (Thread-14) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:01:27 ERROR (Thread-7) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:01:38 ERROR (Thread-10) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:01:49 ERROR (Thread-18) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:00 ERROR (Thread-8) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:11 ERROR (Thread-13) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:22 ERROR (Thread-4) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:33 ERROR (Thread-9) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:44 ERROR (Thread-17) [DenonAVR] No mapping for source MPLAY 2017-07-07 10:02:55 ERROR (Thread-8) [DenonAVR] No mapping for source MPLAY

@Zippit This PR solved the issue.

@bieniu Thanks! I'll wait for the next release then. 😄


EPIC-HP as a Hop-by-Hop extension option

Currently, EPIC-HP is implemented as a separate path type (PathType field value 3). As described in the documentation, this path type is defined by four 4-byte fields (timestamp, counter, and and two hop verification fields), followed by the normal SCION path.
This could be implemented as a Hop-by-Hop extension option and using the normal SCION path type. The extension option would contain the additional fields. The routers two last routers on a path could, if they choose to implement EPIC-HP, check if the extension option is present and process it. Any other router just processes the normal SCION path.
The advantages/disadvantages are:

:heavy_minus_sign: 4 byte overhead for Hop-by-Hop extension header and TLV option header
:heavy_minus_sign: more work for downstream systems (duplicate suppression system) to extract the EPIC-HP fields (search for option type)
:heavy_plus_sign:  fewer path types to support (in routers, applications, tools, etc.)
:heavy_plus_sign:  EPIC-HP supported in transit routers without explicitly implementing the EPIC-HP path type

One additional point is the handling of reverse paths with EPIC-HP; for reversing a path with EPIC-HP path type, additional information would be required to add valid HVFs for the reverse direction. As this is not usually available, the reverse path is "downgraded" to a normal SCION path instead.
An EPIC-HP implementation based an extension option would handle this more naturally. The reverse path would just be the normal SCION path and a reply packet would naturally not include the extension option, unless it is explicitly added.
Summary
EPIC-HP appears to be ideal for an implementation in an extension header option, as it just adds additional fields that need to be processed by some routers along the path. Yes, compared to a separate path type, there is some overhead in terms of header size and parsing, but perhaps the point is; If the hop-by-hop extension is not good enough for this use case, when would we ever use it?

Thanks for the reply @sgmonroy.

I think you forgot to mention that now all routers in the path need to check whether they need to process the HBH extension or not.

This is not accurate. Only the routers last two routers along the path, i.e. those that want to process the option to verify the EPIC fields, need to check for the existence of the corresponding HBH option.
It would be correct to say that any router that wants to process some other HBH extension option would need to skip the EPIC option while looking for the options it's actually interested in. But I don't think it would be fair to attribute this extra effort of skipping to the EPIC HBH option, and as we currently don't have any HBH option support it's currently just not an issue.
I don't quite see a reason why HBH Extensions should be reserved for use cases where every router along the path needs to process the options.
If what I described above is not sufficient, because we do not want the overhead of checking for the existence of the HBH option even for the last two routers, I think one additional option to your suggestions would be to use the router alert flag; we could generally say that HBH are only processed by routeres if the alert flag is set.

With the newly proposed EPIC-SAPV system (see #4110), with yet another path type, I think having this discussion is still very relevant.
The main argument in favor of using a separate path type for this is that this allows to cut off and re-use four bytes of the hop-field-MAC field, which is arguably a neat trick.
However, defining new path types will incur a cost at the entire SCION "ecosystem". Every SCION component, routers, libraries, applications, tools, middleboxes,etc. MUST implement these path types in order to be even minimally compatible. The system making use of a separate path type can only be usable if the path type is supported by every component along a path. The designated extension mechanism avoids these pitfalls. Extension options can always easily be skipped and the default processing for the packet can be unaffected. Importantly, systems making use of extension options will naturally have to deal with partial support for a new option, making it more likely that even a partially deployed system can be useful.
I'm not saying that we should never define new path types -- being able to do so appears like a very powerful option. However, I think the very strong default for new systems should be to rely on the extension header mechanism as far as possible. Once this has been explored, and the system is well understood and all the trade offs are clear, and there are strong disadvantages from using extension headers, then maybe adding a new path type can be considered.
P.s. Btw. this also applies to Colibri, but that discussion is just so much more complicated...

@matzf I think @sgmonroy provided a very nice alternative to what you suggest:

Provide HP as a next header and similarly to the E2E, have routers with HP enabled to only accept packets with next header HP.

pros: no router updates needed for routers along the path, no extra parsing.
cons: could be argued it should not be a "protocol" on its own.

I would actually for such a solution and agree that if we don't really need to create a new path type we should not.

@lukedirtwalker Thanks for your reply. I don't remember why I didn't reply to this proposal specifically, but I agree, this looks like a better solution than the separate path type.
In essence, this would amount to defining a separate extension header type. Is your idea that this new extension header would then also contain TLV encoded options, or would this be specific to EPIC? Can you explain why you would think this is more appropriate than putting the EPIC data as an option into the existing Hop-by-Hop extension header?
I had always been thinking of the HBH-extension as quite generic container for all kinds of extension options to be processed by routers -- do you have something more specific in mind, then?  Do you have some examples for options that you should be added to the HBH extension (there are none defined yet)?


Allow configuring the finder in the plugin setup

🚀 The feature, motivation and pitch
:ObsidianSearch is trying - in that order - Telescope, fzf-lua, and fzf.vim:
command.search = function(client, data)
  local base_cmd = vim.tbl_flatten { util.SEARCH_CMD, { "--smart-case", "--column", "--line-number", "--no-heading" } }

  local has_telescope, telescope = pcall(require, "telescope.builtin")

  if has_telescope then
    -- ...
    return
  end

  local has_fzf_lua, fzf_lua = pcall(require, "fzf-lua")

  if has_fzf_lua then
    -- ...
    return
  end

  -- Fall back to trying with fzf.vim
  local has_fzf, _ = pcall(function()
    -- ...
  end)

  if not has_fzf then
    echo.err "Either telescope.nvim, fzf-lua or fzf.vim is required for :ObsidianSearch command"
  end
end

I prefer fzf-lua, but because Telescope is the standard I have it installed for plugins that don't support anything else, so Obsidian will use Telescope on my machine because it's the first in the tryout order.
I would like an option to configure, with a setting, which option ObsidianSearch will use. Something like:
require'obsidian'.setup {
    finder = 'fzf-lua',
}

Alternatives
No response
Additional context
No response

Hey @idanarye, definitely makes sense! Happy to review a PR here.

Closed as completed.


fix: Adapt Release train endpoint to take commitHash as optional input

Release Train now run release train to specific commit hash instead of always to latest changes
REV: DSN-BADARR

Please change "feat" to "fix" when merging (I did it for the title), because this is not a new feature for a kuberpult user/operator.


Don't allow Apache to serve git files

This is a fairly minor change to .htaccess - but Apache should not serve git files or directories to users.

I don't think that this configuration should be a part of Lychee. It should be part of your server configuration instead.


bctls: Nullpointer exception in TlsServerProtocol on renegotiation

We integrated bctls as Provider in our TLS-Server (jetty 9.4.31.v20200723, protocol version TLSv1.2). When a client tries to renegotiate, the server responds with "SSL alert number 80" which indicates an internal error. On the server side, we see a NullPointerException in the logs.
We reproduced the exception with bc-java/tls/src/test/java/org/bouncycastle/tls/test/TlsServerTest.java. There, we observe that in the class  TlsServerProtocol at the line 873 the server tries to access an object which is null at that moment. The expected behaviour is that the server checks the variable for a valid value and returns an appriopriate message.
Stacktrace:
TLS server raised alert: fatal(2), internal_error(80)
> Failed to read record
java.lang.NullPointerException
    at org.bouncycastle.tls.TlsServerProtocol.handleHandshakeMessage(TlsServerProtocol.java:873)
    at org.bouncycastle.tls.TlsProtocol.processHandshakeQueue(TlsProtocol.java:620)
    at org.bouncycastle.tls.TlsProtocol.processRecord(TlsProtocol.java:516)
    at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:232)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
    at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
    at org.bouncycastle.tls.TlsInputStream.read(TlsInputStream.java:29)
    at org.bouncycastle.util.io.Streams.pipeAll(Streams.java:114)
    at org.bouncycastle.tls.test.TlsServerTest$ServerThread.run(TlsServerTest.java:64)
Exception in thread "Thread-74" java.lang.RuntimeException: org.bouncycastle.tls.TlsFatalAlert: internal_error(80)
    at org.bouncycastle.tls.test.TlsServerTest$ServerThread.run(TlsServerTest.java:69)
Caused by: org.bouncycastle.tls.TlsFatalAlert: internal_error(80)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:796)
    at org.bouncycastle.tls.TlsProtocol.readApplicationData(TlsProtocol.java:731)
    at org.bouncycastle.tls.TlsInputStream.read(TlsInputStream.java:29)
    at org.bouncycastle.util.io.Streams.pipeAll(Streams.java:114)
    at org.bouncycastle.tls.test.TlsServerTest$ServerThread.run(TlsServerTest.java:64)
Caused by: java.lang.NullPointerException
    at org.bouncycastle.tls.TlsServerProtocol.handleHandshakeMessage(TlsServerProtocol.java:873)
    at org.bouncycastle.tls.TlsProtocol.processHandshakeQueue(TlsProtocol.java:620)
    at org.bouncycastle.tls.TlsProtocol.processRecord(TlsProtocol.java:516)
    at org.bouncycastle.tls.RecordStream.readRecord(RecordStream.java:232)
    at org.bouncycastle.tls.TlsProtocol.safeReadRecord(TlsProtocol.java:768)
    ... 4 more

Components used for the test:
Client:

OpenSSL version 1.1.1f
testssl.sh version 3.0

Server:

BouncyCastle version 1.67
OpenJDK version <IP_ADDRESS>

Reproduce the exception:

Run TlsServerTest.java
-- default port: 5556
run openssl s_client:
-- $ openssl s_client -connect <IP_ADDRESS>:5556 -reconnect -no_ticket
Type "R" and return to renegotiate
-- output: [...]: SSL alert number 80

To test with testssl.sh run:
$ ./testssl.sh -R <IP_ADDRESS>:5556
-- The results are displayed green, but server throws NullPointerException
If you need any further Information please let me know.

This should be fixed in 168b01, available from https://downloads.bouncycastle.org/betas/ . However please note that our TLS implementation does not support renegotiation, other than to refuse it via a no_renegotiation alert.

okay thank you for the update, i forgot to check the current betas, i will try to reproduce it there.
The expected behaviour is no_renegotiation, correct. I just wanted to report the server side Exception.

So the current master HEAD 66d368d still shows the same exception and the beta bctls-jdk15on-168b01 ,where i am not sure which revision this correlates to, produces the same issue.

@flako-dd  I'm just revisiting this and cannot see the exception using the proposed openssl renegotiation test. Are you still seeing this issue (on say 1.69 or the latest beta - 169b04 at present)?


ScrollMode.ALWAYS probably works incorrect

Description

If I activate scrolling for a column and set ScrollMode.ALWAYS and at the same time add only one or some elements to the column whose height is less than the height of the column itself, then the scrollbar is not displayed.
Code example to reproduce the issue:
import flet

def main(page:flet.Page):
    page.title = "Scroll Column Test"
    column = flet.Container(
        content=flet.Column(
            width=200,
            height=200,
            scroll=flet.ScrollMode.ALWAYS,
            # controls=[flet.Text(value=str(i)) for i in range(100)]
            controls=[flet.Text(value="single string",height=20)]
            ),
        theme=flet.Theme(
            scrollbar_theme=flet.ScrollbarTheme(
                track_color={
                    flet.MaterialState.DEFAULT: "#E3E7E8",
                },
                track_visibility=True,
                thumb_color={
                    flet.MaterialState.DEFAULT: "#616161",
                },
                radius=10,
                main_axis_margin=10
            )
        )
    )
    page.add(column)

if __name__ == "__main__":
    flet.app(target=main)

Describe the results you received:

Vertical scrollbar is not displayed
Describe the results you expected:
Vertical scrollbar should be visible, but inactive

Flet version (pip show flet):
Name: flet
Version: 0.22.0
Summary: Flet for Python - easily build interactive multi-platform apps in Python
Home-page:
Author: Appveyor Systems Inc.
Author-email<EMAIL_ADDRESS>License: Apache-2.0
Location: C:\Users\HermanC\PycharmProjects\flet-Learn\venv\Lib\site-packages
Requires: cookiecutter, fastapi, flet-runtime, packaging, qrcode, uvicorn, watchdog
Required-by:

Give your requirements.txt file (don't pip freeze, instead give direct packages):
flet==0.22.0

Operating system:

Windows 11 21H2

Looks like same issue, but for ScrollMode.HIDDEN - https://github.com/flet-dev/flet/issues/2388

Hello team, problem was resolved in new flet version 0.23.0. Thank you!


Nothing shows in Blender 4

I assume this doesn't work in Blender 4 because it's from 2021, right? just wanted to make sure. I downloaded the release, and I installed it, in fact I see all 5 files in ~/Library/Application Support/Blender/4.0/scripts/addons, but inside Blender it doesn't show at all, and LWO is not a format in the import menu.

For blender add-ons you copy the full zip files (IE: Don't uncompress them) into your add-ons directory then you add and activate it in the application - the LWO option will show up under the import menu..


Fix for MIFOSX-2799

Issue:
When a new user is created in the community-app, the E-mail received on registration was too complicated to understand
Fix:
Changed the E-mail in a more understandable way.

The subject line should be "New User Information" or 'Welcome to [Organization] Name'. I meant [Organization] should be replaced with Organization name.  Remove [ ]. Sorry that I was not clear enough in previous review comment.  :-(

Yeah thats fine, I'll send new PR


Add LegacyStrings for julia 0.5 compatability

This should make things run on current julia 0.5 builds.

Coverage decreased (-58.3%) to 0.0% when pulling 7c23be665f134ba28d5b1965cc6e4fc4e658537e on davidanthoff:legacystrings into 494f8316a868a24bbc09aa2626f0685ec939cf8e on stevengj:master.

I just realized that the travis config still relies on the staticfloat PPA, which is not maintained and won't give us a recent julia 0.5 build. I think that explains the CI errors. I'll try to figure out what to do about the travis stuff.

This should not be needed with PyCall master, thanks to cd80a6e5fac0e3e70f72fb9bef87bfdbf354ab6b

See also #295

Ah, I had tried master but hadn't run Pkg.build.

@stevengj Could you tag a new version? Would be really helpful to have a tagged version that runs on 0.5 RC0 so that downstream packages can more easily be brought in shape for 0.5. If that version still has deprecation warnings, no problem, but having something that doesn't error out would be great.

Yeah, I'm planning on tagging new versions of a bunch of my packages in the next couple of days.

Cool. I think PyCall especially would help. The downstream packages really need a tagged version that works on travis and appveyor to get their julia 0.5 porting working. I actually don't need a version that doesn't throw deprecation warnings, from my point of view just a tag on the current master would already help a great deal. The deprecation warnings could be fixed later, they don't really stop downstream packages from getting their stuff in order.

JuliaLang/METADATA.jl#5793


Added the code editor beta version

CSS is a bit off. Requires polish

Closing this until further progress is made on this


Quick Stats Link for Embedding in Site

⚠ Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find similar feature request

🏷 Feature Request Type
UI Feature
🔖 Feature description
Is it possible to have the Quick States available for embedding in an iFrame ?
Would be great to be able to embed into another web site for quick overview
✔ Solution
iFrame availability of Quick Stats for be embedded in another web site.
Anonymous connection
❓ Alternatives
No response
📝 Additional Context
No response

It would be great if this wasn't specifically an iframe. That's a bit limiting.
I'm looking for ways to get that summary data like All Systems Operational, Partially Degraded, but I'd like to be able to style it, etc. to match the client that it would be displayed in.
Maybe this could be an GET endpoint that returns a json payload? Just thinking about the API being discussed in #118

I think a quick stats api would be great so anyone can embed the data to anywhere they like.
An example would be:
GET https://uptime.example.com/api/quick-stats

And then return something like:
{
    "up": 143,
    "down": 2,
    "maintenance": 0,
    "unknown": 0,
    "pause": 4
}

Look at https://status.yourdomain.xyz/api/status-page/your-status-page-name. It returns a JSON of all the added status monitors, but if you need heartbeat, look at https://status.yourdomain.xyz/api/status-page/heartbeat/your-status-page-name.

Look at https://status.yourdomain.xyz/api/status-page/your-status-page-name. It returns a JSON of all the added status monitors, but if you need heartbeat, look at https://status.yourdomain.xyz/api/status-page/heartbeat/your-status-page-name.

Is there an easy way just to see if the current overall status of the services is degraded/operational? Without having to fetch the entire heartbeat history of all the monitors.

Thanks for the tip. Unfortunately, the links do not give any information about the status of the individual monitors. even if I evaluate the time series, paused monitors continue with the output of the last status.
The suggestion of @ianpogi5 will be a good solution for integrations. Additionally, there is no need to remember to add new monitors to a status page.
If this becomes too complex, an output with the last status of all monitors would be good, so that you can easily count and evaluate yourself. This will be my preferred solution.


Wrong path to .venv directory in guides

Hi!
As an examle:
./venv/bin/pip install -r requirements.txt
Should be:
./.venv/bin/pip install -r requirements.txt
And instructions for sentinel is crontab is wrong too.
Should be:

cd /home/user/sentinel && ./.venv/bin/python bin/sentinel.py >/dev/null 2>&1

Oh, sorry it was my own fault..
I wrote
virtualenv .venv
Instead of
virtualenv ./venv
So I created directory .venv instead of venv
Sorry, I close this issue.


publish intermediate data

Enable bcfishpass processing on memory limited systems by posting intermediate tables that are fairly static.
Scripts for prepping these might actually be better held in fwapg or a stand alone repo, they are useful for a lot more than fish habitat modelling:

discharge
channel width
precipitation

Easily posted:
https://bcfishpass.s3.us-west-2.amazonaws.com/channel_width.csv
https://bcfishpass.s3.us-west-2.amazonaws.com/discharge.csv
https://bcfishpass.s3.us-west-2.amazonaws.com/mean_annual_precip.csv
Next step is modifying the build to default to loading these rather than computing.

closing in favour of https://github.com/smnorris/bcfishpass/issues/344


OCPBUGS-24482:  Enable new features to prevent CSI driver from crash looping (revert OCPBUGS-23970)

Reverts openshift/vmware-vsphere-csi-driver-operator#192

/retest

/jira refresh
/retest

/jira refresh

/cherry-pick release-4.15

/retest-required

/label qe-approved

/retest-required

/hold cancel
/override ci/prow/e2e-vsphere-ovn-upgrade
because of https://issues.redhat.com/browse/OCPBUGS-25431

/override ci/prow/e2e-vsphere-ovn-upgrade


add strawberry.relay tests, improve compatibility with relay

Fix partly the compatibility with strawberry.relay
The part which keeps the schema validation crashing is:
the graphql name of the _id field is ignored. Instead ´_id´ is transformed to Id even with ``auto_camel_case` off, This causes the interface validation to crash.
Seemingly strawberry-graphql-django just ignores graphql names and need some updates

Description

Types of Changes

[ ] Core
[x] Bugfix
[ ] New feature
[ ] Enhancement/optimization
[ ] Documentation

Issues Fixed or Closed by This PR
Checklist

[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[x] I have added tests to cover my changes.
[ ] I have tested the changes and verified that they work and don't break anything (as well as I can manage).

And the classvar stuff is a little hacky but seemingly there is no better solution


iOS data not being sent or received

The Parse team asked me to log this bug under the iOS SDK. I found an issue where custom data is not sent or received using the Parse push notifications. The code I'm sending from one iOS device to another iOS or android device results only in a result where the Alert message can be viewed. No data is passed along. Please fix this SDK bug asap. Thanks.
NSDictionary *data = @{
                       @"email"<EMAIL_ADDRESS>
                       };

PFPush *push = [[PFPush alloc] init];
PFQuery *pushQuery = [PFInstallation query];
[pushQuery whereKey:@"device_id" equalTo:@"1234567890"];

[push setQuery:pushQuery]; // Set our Installation query
[push setData:data];
[push setMessage:[NSString stringWithFormat:@"John Doe requests to connect with you."]];
[push sendPushInBackground];

Hey @bryanchoi, are you referencing setData: portion of the code above?
If yes - then it looks like it works according to the documentation on the method: @warning This will overwrite any data specified in setData..
I recommend rewriting your code into something like this, this should send both alert and email and is close to what you would send via REST API or Parse Dashboard.
NSDictionary *data = @{
                       @"alert" : [NSString stringWithFormat:@"John Doe requests to connect with you."],
                       @"email"<EMAIL_ADDRESS>};

PFPush *push = [[PFPush alloc] init];
PFQuery *pushQuery = [PFInstallation query];
[pushQuery whereKey:@"device_id" equalTo:@"1234567890"];
[push setQuery:pushQuery]; // Set our Installation query
[push setData:data];
[push sendPushInBackground];

Let me know if it helps, or feel free to reopen this issue if there is anything I might have missed.

Nikita, thank you very much for that clarification. This helped resolve
this issue and now I'm able to see the custom data including the alert
message. Much appreciated!
On Mon, Jan 25, 2016 at 12:44 AM, Nikita Lutsenko<EMAIL_ADDRESS>wrote:

Hey @bryanchoi https://github.com/bryanchoi, are you referencing
setData: portion of the code above?
If yes - then it looks like it works according to the documentation on the
method: @warning This will overwrite any data specified in setData..
I recommend rewriting your code into something like this, this should send
both alert and email and is close to what you would send via REST API or
Parse Dashboard.
NSDictionary *data = @{
@"alert" : [NSString stringWithFormat:@"John Doe requests to connect with you."],
@"email"<EMAIL_ADDRESS>};
PFPush *push = [[PFPush alloc] init];
PFQuery *pushQuery = [PFInstallation query];
[pushQuery whereKey:@"device_id" equalTo:@"1234567890"];
[push setQuery:pushQuery]; // Set our Installation query
[push setData:data];
[push sendPushInBackground];
Let me know if it helps, or feel free to reopen this issue if there is
anything I might have missed.
—
Reply to this email directly or view it on GitHub
https://github.com/ParsePlatform/Parse-SDK-iOS-OSX/issues/783#issuecomment-174410491
.

Hi Nikita,
I'm seeing issues where some push notifications in my beta test build which
test users are using are not being sent. All the devices I own and in my
testing work fine, but then sporadically pushes are not getting
sent/received on test users' devices. When I click into it, it says, APNS -
invalid token. The curious thing is if the other user running the beta test
build has the app open it handles the notification. It's only when the app
is minimized, he sometimes gets the message or not. Please advise as
getting this running consistently is critical to the functionality of my
app.
Bryan
[image: Inline image 1]
On Mon, Jan 25, 2016 at 11:34 AM, Bryan Choi<EMAIL_ADDRESS>wrote:

Nikita, thank you very much for that clarification. This helped resolve
this issue and now I'm able to see the custom data including the alert
message. Much appreciated!
On Mon, Jan 25, 2016 at 12:44 AM, Nikita Lutsenko <
<EMAIL_ADDRESS>wrote:

Hey @bryanchoi https://github.com/bryanchoi, are you referencing
setData: portion of the code above?
If yes - then it looks like it works according to the documentation on
the method: @warning This will overwrite any data specified in setData..
I recommend rewriting your code into something like this, this should
send both alert and email and is close to what you would send via REST API
or Parse Dashboard.
NSDictionary *data = @{
@"alert" : [NSString stringWithFormat:@"John Doe requests to connect with you."],
@"email"<EMAIL_ADDRESS>};
PFPush *push = [[PFPush alloc] init];
PFQuery *pushQuery = [PFInstallation query];
[pushQuery whereKey:@"device_id" equalTo:@"1234567890"];
[push setQuery:pushQuery]; // Set our Installation query
[push setData:data];
[push sendPushInBackground];
Let me know if it helps, or feel free to reopen this issue if there is
anything I might have missed.
—
Reply to this email directly or view it on GitHub
https://github.com/ParsePlatform/Parse-SDK-iOS-OSX/issues/783#issuecomment-174410491
.


Fingerprint Database

Hello,
are there any plans to start a database of known fingerprints?

Hi @hazcod, apologies for the delay, There are groups such as Shodan and GreyNoise who do make hasshServer available to search. We'd be interested to hear if anyone is collecting and sharing hassh data. Those related to internet scanners (collected by sensors) would be interesting to see shared. One issue that should be considered is that information would likely be shared just like with most Indicators - on an "all care taken, use at own risk" basis.


Trivy and Grype scans show the following CVEs

We run trivy and grype scan on our images and since we use calico for our CNI those images also get scanned.
image calico/cni scanned.
NAME                            INSTALLED                                           FIXED-IN                           TYPE       VULNERABILITY        SEVERITY
github.com/emicklei/go-restful  v2.11.2-0.20200112161605-a7c079c43d51+incompatible  2.16.0                             go-module  GHSA-r48q-9g5r-8q2h  Critical
golang.org/x/sys                v0.0.0-20210809222454-d867a43fc93e                  0.0.0-20220412211240-33da011f77ad  go-module  GHSA-p782-xgp4-8hr8  Medium
image calico/pod2daemon-flexvol scanned.
NAME                             INSTALLED  FIXED-IN  TYPE       VULNERABILITY        SEVERITY
github.com/projectcalico/calico  (devel)              go-module  CVE-2020-13597       Low
github.com/projectcalico/calico  (devel)    0.8.6     go-module  GHSA-pf59-j7c2-rh6x  Medium
Expected Behavior

Can the dependencies be upgraded to get the scan to be cleaner. There will be some that cant be fixed easily - but these seem like they can be.
Current Behavior

Possible Solution

Steps to Reproduce (for bugs)

just run trivy or grype scan on the container images and it will show these errors

Context

Since we ship the packages to a customer for an on-prem deployment we'd like to make sure we minimize the security vulnerabilities as much as possible and hence this ask.
Your Environment

Calico version latest 3.24 release
Orchestrator version (e.g. kubernetes, mesos, rkt): k3s 1.24
Operating System and version: Ubuntu 18.04
Link to your project (optional):

@Behnam-Shobiri Can you please take a look?

Thank you for reporting the CVEs @samirsss
Regarding CVEs in calico/pod2daemon-flexvol

They both have the same CVE (GHSA-pf59-j7c2-rh6x is the GitHub ID of CVE-2022-29526). This is a False Positive (we are not vulnerable). As mentioned in the description of this CVE "Calico (version 3.14.0 and below)" are vulnerable. Since you are using the latest version of 3.24 -> you are not vulnerable to this CVE (not sure why the scanner is picking it up).

For CVEs in calico/cni:

CVE-2022-1996: As I mentioned in https://github.com/projectcalico/calico/issues/6733; this is an indirect dependency of K8s and we are not affected by it. This has been removed in 3.25.1; we do encourage everyone to use the latest version of Calico to get all the security fixes.
CVE-2022-29526: This is an indirect dependency of https://github.com/containernetworking/plugins. The vulnerable function is not used directly. But again we do encourage everyone to use the latest version of Calico to get all the security fixes. (we are using a newer version of golang  -> this CVE is fixed).

Hope this can help

@Behnam-Shobiri thank you for the detailed clarification here. Much appreciated

Closing this since it seems this is resolved.


Fire canplay event when YouTube iFrame API onReady event is fired

This is a very simple change but would help me immensely. Would it be possible to fire a canplay event when the YouTube iFrame API onReady event is fired?
For example, add these two lines:
// fire canplay event
mejs.YouTubeApi.createEvent(player, pluginMediaElement, 'canplay');
at ...
https://github.com/johndyer/mediaelement/blob/master/build/mediaelement-and-player.js#L1720
I'm trying to take action as soon as the duration is known and just this simple change allows me to use a canplay event to do this across all sources.

@dazweeja Thanks for pointing this. I'll create a PR integrating this, given that it could be useful for others having an event available at that stage

@dazweeja I created #1802 to fix this. Feel free to check it and let me know if it works for you

Looks good to me. Thanks!

Closing issue since fix has been merged


Check if the frame is writable before copyPlanes

For safety of frame operations, check if the frame is writable before copyPlanes.
If this seems to be a performance overhead, I would like input as this PR may not be necessary.

Thanks for the PR! ❤
Let me know whether you need a tag 👍

Please tag this as well 🚀

I've created a v0.23.0 tag 👍


Update Readme to ctrace-js

ctrace should be labeled ctrace-js in this context

Coverage remained the same at 100.0% when pulling d5ad6114a58deb85ee0e18218fdd2bfbdd6799ed on schlank:patch-1 into ccebe89a422ff6ffd15a6e78c0402477fe95f929 on Nordstrom:master.

Coverage remained the same at 100.0% when pulling d5ad6114a58deb85ee0e18218fdd2bfbdd6799ed on schlank:patch-1 into ccebe89a422ff6ffd15a6e78c0402477fe95f929 on Nordstrom:master.

Coverage remained the same at 100.0% when pulling d5ad6114a58deb85ee0e18218fdd2bfbdd6799ed on schlank:patch-1 into ccebe89a422ff6ffd15a6e78c0402477fe95f929 on Nordstrom:master.


Multi-world info message has a typo

https://gist.github.com/Mahagon/e064b6a323e9da27e491#file-gistfile1-txt-L20
i hope its worth being reported^^
Its just a small typo
:)

Whoops XD. Will cover it in next push.

@Minecrell
Are you getting the double printout for Preparing ... locally?

@Zidane Any news on these word changes you made? Did you already push them?


How to trigger registerDiagnosticsPushSupport function in html/css/json-language-server.

Does this issue occur when all extensions are disabled?: No

VS Code Version: 1.89.0

I use monaco-editor, monaco-languageclient and css-language-server which is build-in in vscode to implement css lsp feature. The client's capabilities include the textDocument.diagnostic option, and this item cannot be removed. Causes css-server to execute the registerDiagnosticsPullSupport method, but I want to use the registerDiagnosticsPullSupport method.
How can i do?
// /vscode/extensions/css-language-features/server/src/cssServer.ts
const supportsDiagnosticPull = getClientCapability('textDocument.diagnostic', undefined);
if (supportsDiagnosticPull === undefined) {
	diagnosticsSupport = registerDiagnosticsPushSupport(documents, connection, runtime, validateTextDocument);
} else {
	diagnosticsSupport = registerDiagnosticsPullSupport(documents, connection, runtime, validateTextDocument);
}

textDocument.diagnostic means client supports pull. Pull is preferable.
You are implementing a client?

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Additional "sensors"

I love the work put into this integration and so far it works like a charm!
I was wondering if it was possible (and reasonable) to add some additional sensors, such as:

Estimated driving range
Charge status (something like, Not chargning | Charging | Scheduled at xx
Charging speed
(- Location as coordinates)

Would you like these to be visable in the Homey app? Or use them as variables in flows?

At least for me, the driving range would be useful in the app and the charge status would be mostly useful as variable (charging speed as well I guess).


Lungs Segmentation | long term

The initial lungs segmentation method has been provided by the PR #133 due to the issue #120. The provided approach, however, can be further improved e.g., by reducing its trend to false positives and made the algorithm more stable. Furthermore, the ability to correctly separate the tissues such as bones, lungs, bronchial, etc. will be beneficial for a further work with data augmentation.
Possible Solutions
For example, the work of van Rikxoort et al. describes the automatic error detection method via the convex hull complement to a coastline of lungs:

Furthermore, the method provided by S. Hu et al. is aimed at junction line enhancement followed by lungs separation which I've found to be unreasonable resource consumptive though.

The ability of the bronchial / lungs separation described in the paper of T. Kitasaka et al. I guess will also be valuable as an additional instrument of data augmentation.

Volume rendered results of the ground truth (left), the proposed method (middle) and the adaptive branch tracing method (right). Yellow region indicates TP while blue region shows FP Though the bronchial lumen is interrupted by a tumor indicated by the arrow, the proposed method can extract the lumen regions beyond the tumor.

P.S. Of course, contributions like a radiologist's handcrafted annotations of anatomical structures inside CT volume will be appreciated.
Current Behavior
The lungs segmentation lied in the preprocess/lung_segmentation.py consists of the followed steps:
Step 1: Convert into a binary image.
Step 2: Remove the blobs connected to the border of the image.
Step 3: Label the image.
Step 4: Keep the labels with 2 largest areas.
Step 5: Erosion operation with a disk of radius 2. To separate lung nodules attached to blood vessels.
Step 6: Closure operation with a disk of radius 10. To keep nodules attached to a lung wall.
Step 7: Fill in the small holes inside the binary mask of lungs.
Acceptance criteria
The good measure of a lungs segmentation algorithm should be Hausdorff distance which has efficient implementation by scipy.spatial.distance.directed_hausdorff, thus the acceptance criteria will rely on the Hausdorff tests. Nonetheless, this is the long-term issue, thus different type of contribution may be accepted by this issue.

Yep, we're definitely supposed to refine the lung segmentation. Have a look at e.g. patient 0001 and the nodule at<PHONE_NUMBER>1,<PHONE_NUMBER>7, -116.12078059 (slice ~91):

It's state is labeled as solid, which isn't included in the current lung segmentation:

However, some of the slices of the nodule are included in the lung segmentation but currently this would still falsify statistics such as the predicted volume of a nodule as to be implemented in #3 .

As part of the evaluation, the Luna16 data include right and left segmented lungs with trachea which could serve as a good baseline


Machine mac-4 lost external connection from iPhone device.

Machine mac-4 lost external connection from iPhone device.

External connection has been restored by rebooting the machine.


Support "react-native" package entry point

Adds conditional export for react-native environment.
This fixes UUID not working in Jest with the react-native preset. Before this change it was falling back to default and then failing to find the global crypto.
react-native inserted after browser so React Native for Web can still be targeted properly, see https://nodejs.org/api/packages.html#community-conditions-definitions for more info.

Oh, and I should have explained, since RN 0.71 there's a custom environment for Jest which extends NodeEnv: https://github.com/facebook/react-native/commit/cb2dcd327c62edae58bb8571273add4c286fd2a8#diff-70258ed063b4b3f0c6a81e698a3f928cd310f4782e8c260e08900809f5cfa9e4

there's a custom environment for Jest which extends NodeEnv

Naive question:  Is this an issue with that custom environment, then?  E.g. the "react-native" configuration you've added is identical to the "node" configuration.  Could jest not fallback to the "node" configuration here in that case?
(Asking because I fundamentally believe that adding complexity to address context-specific issues in downstream dependencies is a pretty serious code smell.  We do it, but mostly because @ctavan is a much more understanding and gracious maintainer than I am. :-) )
Also, would it be possible to add a test-case for this?  I'm not familiar with react-native so I don't know what that would look like, but it would be helpful to have code that demonstrates the problem.

Could jest not fallback to the "node" configuration here in that case?

Interesting question! I don't know, let me do some more research on the RN/Jest side and see if what you've suggested is possible...

For anyone finding this issue in future, the relevant PR on the RN project is here: https://github.com/facebook/react-native/pull/39237


Add Rubocop to parser documentation

I noticed Rubocop has a parser, but it's not documented.
https://github.com/DefectDojo/django-DefectDojo/blob/dev/dojo/tools/rubocop/parser.py

@rc-mattschwager Thanks for helping keep our docs honest/up to date :+1:


bull-board in pm2 cluster mode

Hi guys,
it seems bull-board doesn't work in pm2 cluster mode. This issue is not the same as #33 or #48 i think - these two issues kind of mix up redis cluster mode and pm2/node cluster mode. But in our case i mean pm2 cluster (just for clarification).
Here is the problem: bull-board works just fine. But in production a http request is handled from one out of 8 processes due to the cluster mode. All bull queues are registered in the master process. So if the request to GET /admin/queues/queues/? is handled by the master process the queues and jobs show up. If the http request is handled by any of the other 7 node processes the queues don't show up. This means bull-board shows "No stats to display" most of the time.
Is there a way to fix this? If yes i'm happy to contribute with a little hint, what to change ;)
Thanks and best regards

In your case I would recommend to simply forward the bull-board endpoint to your master worker instead of balancing them between all your workers, or you need to register the bull queues to bull-board on all your workers.

Thanks @Heartz66 for your quick reply. I decided to register the queues on all workers. One thing to note is, that in this case one has to make sure that the jobs in the queue are still completed one after one and not in parallel. If jobs are not completed one after one any more, it is most likely due to the fact, that the process function is triggered from every worker. You can fix this with the following:
const os = require('os');

if (process.env && process.env.pm_id) {
    if (process.env.pm_id % os.cpus().length === 0) {
        const processorPath = path.join(__dirname, 'processors', 'job.js');
        this.queue.process(1, processorPath);
    }
}

This registers the process function only on the node master process. Thanks for the support!


how we can use multiple ipc services using this structure ?

using this structure how do we manage multiple IPC services? because I am using SQLite as db, I need multiple ipc services for module vise crud operation.

Hi @MMJ184, what do you mean by "IPC services" ?
Happy to help, but before going any futher I would like to be sure about your question.
Could you please elaborate your needs with more details and examples.

Hi, @aadiene,
I need to create an Inventory app in (Electron + angular) with (Sqlite + Typeorm) with a repository pattern.
So, I require many ipcMain channels to perform database operations and get data right.
In this structure, you are connecting for data send and receive electron To angular using ipcMain (On WindowApiConst).
but you create only one method for that if anyone requires multiple then how can we manage? can you give any example?

Hi @MMJ184 , do you still need an example of how to implement multiple services ?

Hi @MMJ184 , do you still need an example of how to implement multiple services ?

Yes, if possible please guide me

any newas about this?


Intelisense does not work properly when we work with nested objects\lists

The append methond is not recognized by the intelisense maybe because it is applied to a nested list:

And also as you can see from the picture below, the atributes of edge object are not recognized:

Because the types are hidden within dynamic code, I suspect Pylance can't infer graph is a list of lists. You probably need to add a type hint to help Pylance out: graph: list[list[Edge]] (assuming the list is meant to contain lists of Edge objects).
Tangentially, I would create your graph list as:
[[] for _ in range(nodes)]

For your first assignment in the loop, I would do:
first, second, weight = map(int, input().split())

Thanks @brettcannon. @YordanPetrovDS, please reopen if this is still a problem.


⚠ Fortnite has degraded performance

In b259bf2, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1604 ms

Resolved: Fortnite performance has improved in c5b73c1.


feat: use env var to get node fqdn

Rebase of https://github.com/ockam-network/ockam/pull/1243

Holding on this for now


feat: show a11y features on public site

Issue

Closes #975

Description
(1) Show accessibility features in a bulleted list on the public site under the features category
(2) Show a tag on the image card that the listing is accessible if it has at least one accessibility feature
Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Prototype/POC (not to merge)
[ ] This change is a refactor/addresses technical debt
[ ] This change requires a documentation update
[ ] This change requires a SQL Script

How Can This Be Tested/Reviewed?
Add and remove a11y features from a listing in partners and ensure the bullet points and tags come through
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have reviewed the changes in a desktop view
[x] I have reviewed the changes in a mobile view
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged and published in downstream modules
[ ] I have assigned reviewers
[ ] I have updated the changelog to include a description of my changes
[ ] I have run yarn generate:client if I made backend changes
[ ] I have exported any new pieces in ui-components

Waiting on a design question before finishing the tag feature

Blocked by: https://github.com/bloom-housing/bloom/pull/2531

The long term fix is to move the tags so that if there were multiple tags on the image card they would display properly. Current plan for speed is to merge this as-is with the acknowledgement that there are not any community types right now so at most there is going to be one tag (accessible) showing on a listing card.


Fix pane info in BigQuery load job id

Fixes #28219

Add randomness to job id

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Run PostCommit_Java_Dataflow

Run PostCommit_Java_DataflowV2

There are still multiple code paths relies on the pane info, e.g. write_disposition and create_disposition settings: https://github.com/apache/beam/blob/3ff66d38c41fde475f71254d889ba46440904238/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteRename.java#L194C67-L194C67
let me think more about it. Probably one can save the pane index in WriteResult also

there were test failures due to periodic impulse timing. Re-designed test. If passed then ready to review

Run PostCommit_Java_DataflowV2

Although Dataflow PostCommit have multiple quota exceeded failure, bigquery tests all passed: https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2_PR/218/testReport/org.apache.beam.sdk.io.gcp.bigquery/

there are multiple quota exceeded error for post commit, however bigquery tests all passed: https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2_PR/218/testReport/org.apache.beam.sdk.io.gcp.bigquery/

R: @ahmedabu98
will fix spotless in next iteration

Run PostCommit_Java_DataflowV2

dataflow v2 postcommit passed; java precommit known flink test flakiness. PTAL @ahmedabu98

Run PostCommit_Java_Dataflow

Run PostCommit_Java_DataflowV2

Are you sure this is right? Isn't the purpose of the job id to prevent duplicates?

@kennknowles thanks for confirm. The latest approach is deterministic now (thanks the comment by @ahmedabu98), the WritePartition.Result now preserves the pane index before the ReShuffle happens is used afterwards. The behavior should be totally the same as before on Dataflow Legacy runner, while fixing Dataflow V2.

Java PreCommit known flakiness due to #28383 - might need to disable these tests again

Run PostCommit_Java_DataflowV2

What exactly is this trying to fix? As @kennknowles we do rely on the job id to prevent duplicates (basically we can sometimes reissue a load job and rely on BigQuery to reject the duplicate loads because it has the same job id). We also want to avoid update-incompatible changes here.

After a Reshuffle (we have one before single loads and temp loads), RunnerV2 doesn't propagate pane indices like legacy runner does. Instead of an increasing pane index, RunnerV2 keeps it at 0.
After our reshuffles, we directly rely on pane index to construct a job ID. In the RunnerV2 case the index stays at 0 so in some cases (e.g. single loads) it produces the same job ID for each trigger. The resulting behavior is that only the first load is successful and the rest are seen as duplicates by BQ and ignored.
Yi's solution here is to save the pane index from WritePartitions (a stage that is before Reshuffle) and use that information instead when creating the Job ID in WriteTables.

On Mon, Sep 11, 2023 at 9:02 AM Ahmed Abualsaud @.***>
wrote:

After a Reshuffle (we have one before single loads
https://github.com/ahmedabu98/beam/blob/67e88d9fec9b4ac39a6d42a4bc1e64e1236bc309/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java#L806
and temp loads
https://github.com/ahmedabu98/beam/blob/67e88d9fec9b4ac39a6d42a4bc1e64e1236bc309/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BatchLoads.java#L763),
RunnerV2 doesn't propagate pane indices like legacy runner does. Instead of
an increasing pane index, RunnerV2 keeps it at 0.

This sounds like a bug in Runner V2. Pane indices are supposed to propagate
through shuffles, and we have various things that depend on this.

After our reshuffles, we directly rely on pane index to construct a job ID
https://github.com/ahmedabu98/beam/blob/67e88d9fec9b4ac39a6d42a4bc1e64e1236bc309/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/WriteTables.java#L252-L254.
In the RunnerV2 case the index stays at 0 so in some cases (e.g. single
loads) it produces the same job ID for each trigger. The resulting behavior
is that only the first load is successful and the rest are seen as
duplicates by BQ and ignored.

Should we simply file a bug against RunnerV2?

Yi's solution here is to save the pane index from WritePartitions (a stage
that is before Reshuffle) and use that information instead when creating
the Job ID in WriteTables.
—
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/28272#issuecomment-1714176376, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFAYJVODHIPXMXMVXOIWXZDXZ4YZZANCNFSM6AAAAAA4GWNMFI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Actually each GBK has its own sequence of trigger firings, and when it comes to reshuffle there are two possibilities:

The reshuffle does a GBK then explodes it (per its expansion) in which case the sequence of pane indices come from the Reshuffle/always trigger on that GBK.
The reshuffle is a semantics "no op" at the PCollection level, in which case the pane indices would be preserved from an upstream aggregation, but reshuffle results in bundles from random upstream keys so they are deceptive.

I don't see a way for pane indices crossing a reshuffle to make any sense.

Hi @reuvenlax @kennknowles is there an agreement on the fix?
In any case, we identified there are other issues in FILE_LOAD streaming (#28309) and a fix (#28312) involves changing the coder (and element type) for another GBK used in the pipeline, so we have to break the update compatibility anyways. I can make sure this fix get in as the same beam version as #28312 and note them in CHANGES.md

I'm not sure what you mean by random upstream keys. Reshuffle
currently maps KV -> KV, so no keys are changed. The only difference is
that the keys are now spread out on different workers, which is a semantic
no op.
You may be thinking of Rehsuffle.viaRandomKey, which is a bit different.
On Tue, Sep 12, 2023 at 6:33 AM Kenn Knowles @.***>
wrote:

Actually each GBK has its own sequence of trigger firings, and when it
comes to reshuffle there are two possibilities:

The reshuffle does a GBK then explodes it (per its expansion) in
which case the sequence of pane indices come from the Reshuffle/always
trigger on that GBK.
The reshuffle is a semantics "no op" at the PCollection level, in
which case the pane indices would be preserved from an upstream
aggregation, but reshuffle results in bundles from random upstream keys so
they are deceptive.

—
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/28272#issuecomment-1715736420, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFAYJVLDCEOGYBQCRHRTBSDX2BQDFANCNFSM6AAAAAA4GWNMFI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Right. I was, indeed, thinking of reshuffle via random key. But what I'm saying does apply to Reshuffle by assigned keys. I have been a person who always says that Reshuffle is a semantic noop. If we should preserve that we need the v2 and the SDK implementation to record the pane info (aka reify WindowedValue instead of PaneIndex). This issue has shaken up my beliefs.
If you have Combine -> (stage) -> Reshuffle. Each input to reshuffle has a pane index that corresponds to its key+window in the combine. The main thing is that if you have the original key+window+index you can reassemble the aggregate and/or write it to a sink in a consistent way. But if the stage changes the key you lose that property. Do we even have a good story for what happens to pane info when a DoFn reassigns to new keys?

Notably we only have tests like ReshuffleTest.testReshufflePreservesTimestamps https://github.com/apache/beam/blob/cf0cf3b746c55921ebe4a95d2e06052cb96be74d/sdks/java/core/src/test/java/org/apache/beam/sdk/transforms/ReshuffleTest.java#L115
Considering that reifying and testing against a full windowed value would have be somewhat of a default thing to do, I would guess the original intent was very aware of this issue.

Run PostCommit_Java_DataflowV2

Tests now exercised and passed on Dataflow Runner V2: https://ci-beam.apache.org/job/beam_PostCommit_Java_DataflowV2_PR/226/testReport/org.apache.beam.sdk.io.gcp.bigquery/FileLoadsStreamingIT/

@reuvenlax as @Abacn mentioned, we broke update-compatibility recently in #28312 so that we can unblock dynamic destinations with copy jobs (#28309). Do we have the green light to merge these changes?
If Dataflow folks decide Runner V2 should preserve pane indices, that support may still take some time to implement. In the meantime, this ensures file loads streaming works on the runner (IIUC it previously never did)

FYI - https://issues.apache.org/jira/browse/BEAM-7195 was caused by the
inverse issue. The old code assumed that Reshuffle would produce pane
indices, but of course it just forwards them.
Let's wait until we figure out the correct behavior. Very few users will be
exercising this from runner v2 today, so I would rather be cautious here -
especially if the right answer is to fix runner v2.
On Fri, Sep 15, 2023 at 3:33 AM Ahmed Abualsaud @.***>
wrote:

@reuvenlax https://github.com/reuvenlax as @Abacn
https://github.com/Abacn mentioned, we broke update-compatibility
recently in #28312 https://github.com/apache/beam/pull/28312 so that we
can unblock dynamic destinations with copy jobs (#28309
https://github.com/apache/beam/issues/28309). Do we have the green
light to merge these changes?
If Dataflow folks decide Runner V2 should preserve pane indices, that
support may still take some time to implement. In the meantime, this
ensures file loads streaming works on the runner (IIUC it previously never
did)
—
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/28272#issuecomment-1721050326, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFAYJVINXQC4WCJC6KPYJB3X2QVHVANCNFSM6AAAAAA4GWNMFI
.
You are receiving this because you were mentioned.Message ID:
@.***>

Yea let's sync on this. I think I've come around but I want to get it on dev@ in at least a 1 pager for what the spec is. If it turns out to not be dangerous to reinstate, and we can regain the semantic property "reshuffle is a semantic noop" then that is pretty desirable, so that runners can freely add and remove reshuffles.
Noting that here is where Dataflow v1 implemented reshuffle and re-instated the pre-reshuffle pane: https://github.com/apache/beam/blob/34024902746af90e7bf41e28729ec031dbab58d2/runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/StreamingGroupAlsoByWindowReshuffleFn.java#L61
Noting that Beam's reference implementation, the closest thing we have to a spec, does not preserve the pane but will produce a sequence of increasing panes.

@kennknowles thanks for pin to the related code path for Dataflow legacy worker. I now understand that Dataflow legacy work does some special PTransform override for ReShuffle. For portable pipelines, is that true that there is no longer similar PTransform override, and a possible fix would change java core's ReShuffle to also recover pane (currently it recovers Timestamp) ?
https://github.com/apache/beam/blob/346541b1b824e0343a736ce4e320667b8d73ec06/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/Reshuffle.java#L105

I expect there is a similar override inside the Dataflow service and Unified Worker. The implementation in the SDK would be slow and just for reference, if it was executed. A reshuffle really should not have to cross the Fn API.

But yes if we make the SDK recover the pane info also, then that would make the SDK (direct runner, prism, etc) match Dataflow v1. This would probably not fix v2, but it would fix "the spec".

OK so when I thought about it I agree with everyone else that the reference implementation is wrong and that the right behavior is to be the same as the identity function on PCollections. That means keeping the pane info exactly. I wrote the dev list about it. I think it is probably such a boring thread that it will not get a lot of comment.


Fixes to allow StyleCI to analyze the repo

The next stage is then to apply the fixes StyleCI has provided. StyleCI can do this for you by sending you a PR (and it can keep sending (and optionally merging itself) further PRs if you enable that option).


Custom check for EXT4 Filesystem metrics

We wrote this custom check to export EXT4 filesystem metrics from EC2 instances. The metrics gets tagged with the appropriate PVC name to identify the corresponding kubernetes resource. (It also has other information such as pod, namespace, because the labels are being fetched from the labels on the docker.)
Would be a nice addition as part of an existing integration. Happy to submit a PR in the right place. The code is shown below.
"""
This check reports the following ext4 filesystem metrics

The metrics are obtained by the mount information found in
'/proc/1/mounts' in the datadog container. This is the
same output as running mount command on the host

The filesystem metrics of these devices are found in the
'/sys/fs/ext4/<device_name>' folder

The labels for the metrics are obtained by querying docker daemon for labels
on the container for the pod

Metric Information:
errors_count            number of error counts

lifetime_write_kbytes   shows the number of
                        kilobytes of data that have been written to this
                        filesystem since it was created.

session_write_kbytes    shows the number of
                        kilobytes of data that have been written to this
                        filesystem since it was mounted.

Ref: https://www.kernel.org/doc/Documentation/filesystems/ext4.txt
"""

import docker
import os
from datadog_checks.base.checks import AgentCheck

class Ext4FSMetrics(AgentCheck):

    def __init__(self, name, init_config, instances):
        super(Ext4FSMetrics, self).__init__(name, init_config, instances)

        self.metric_previous_values = dict()
        self.relevant_labels = ['io.kubernetes.pod.namespace']
        self.relevant_metrics = {
            'errors_count': {'metric_type': lambda *args: self.monotonic_count(*args)},
            'lifetime_write_kbytes': {'metric_type': lambda *args: self.monotonic_count(*args)},
            'session_write_kbytes': {'metric_type': lambda *args: self.monotonic_count(*args)}
        }

        self.docker_client = docker.DockerClient(base_url='unix://var/run/docker.sock')

    def check(self, instance):
        pod_fs_metrics_with_labels = self.get_pod_fs_metrics_with_labels()
        self.report(pod_fs_metrics_with_labels, instance)

    def report(self, pod_fs_metrics_with_labels, instance):
        tag_hash = self.init_config.get('tags', {})
        tag_hash.update(instance.get('tags', {}))

        for metrics_with_labels in pod_fs_metrics_with_labels:
            metrics = metrics_with_labels[0]
            labels = metrics_with_labels[1]
            tag_list = [':'.join(pair) for pair in labels.items()]

            for metric, value in metrics.items():
                if metric not in self.relevant_metrics:
                    self.log.error('metric: %s not in relevant_metrics' % metric)
                    continue
                metric_type = self.relevant_metrics[metric]['metric_type']
                metric_type('fs.ext4.' + metric, value, tag_list)

    def get_pod_fs_metrics_with_labels(self):
        """
        Returns the device's ext4filesystem metrics along with pod labels that are using these devices
        grouped by pod_id
        :return: an array of [metrics, labels]
        """
        fs_metrics_with_labels = []
        mount_out_lines = self.get_mount_output()

        for mount_line in mount_out_lines:
            pod_id, device, pvc = self.parse_mount_out_line(mount_line)
            device_fs_metrics = self.get_device_fs_metrics(device)

            labels = self.get_labels_for_pod(pod_id)
            labels['device'] = device
            labels['pvc'] = pvc
            fs_metrics_with_labels.append([device_fs_metrics, labels])
        return fs_metrics_with_labels

    @staticmethod
    def get_mount_output():
        """
        Executes the equivalent of the shell command $(mount | grep volumes | grep ext4)
        :return: ext4 volume mount lines
        """
        with open('/proc/1/mounts', 'r') as reader:
            mount_lines = reader.read()

        ext4_volumes = []
        for line in mount_lines.split('\n'):
            # example of line
            # /dev/nvme3n1 /var/lib/kubelet/pods/0761f982-a52d-4014-aab3-81399a945147/volumes/kubernetes.io~csi/pvc-0fca21ad-bd5a-46ca-b5d3-5c72a4b0e7bd/mount ext4 rw,relatime 0 0
            # ignore sub-paths because they are configmaps
            if '/sub-paths/' in line:
                continue

            if '/volumes/' in line and 'ext4' in line:
                ext4_volumes.append(line)
        return ext4_volumes

    @staticmethod
    def parse_mount_out_line(mount_line):
        """
        Parses the below line

        /dev/nvme2n1 /var/lib/kubelet/pods/857cb797-dc3a-47c6-91dd-43bb85073698/volumes/kubernetes.io~csi/pvc-0fca21ad-bd5a-46ca-b5d3-5c72a4b0e7bd/mount ext4 rw,relatime 0 0
        column 0 is the device name
        column 2 has the pod id and pvc name
        column 2 gets split further by / to get the pod_id and pvc name

        :param mount_line:
        :return: pod_id, device, pvc name
        """
        parts = mount_line.split()
        device = parts[0]
        pod_path_parts = list(filter(None, parts[1].split('/')))
        pod_id = pod_path_parts[4]
        pvc = ''
        for path_part in pod_path_parts:
            if 'pvc-' in path_part:
                pvc = path_part

        return pod_id, device, pvc

    def get_device_fs_metrics(self, device):
        """
        Returns k-v pair of ext4filesystem metrics for device such as /dev/nvme1n1 located at /sys/fs/ext4/nvme1n1
        :param device:
        :return: k-v pair of metrics for each device
        """
        metrics_dir = '/sys/fs/ext4/' + device.split('/')[-1]  # extract nvme1n1 from /dev/nvme1n1
        metrics = []

        with os.scandir(metrics_dir) as files:
            for entry in files:
                if entry.is_file() and entry.name in self.relevant_metrics:
                    metrics.append(entry.name)

        metric_values = dict()
        for metric in metrics:
            with open(metrics_dir + '/' + metric, 'r') as reader:
                metric_value = reader.read()
            if metric not in self.metric_previous_values:
                metric_values[metric] = 0  # do not report giant spike when datadog resets
            else:
                metric_values[metric] = int(metric_value) - self.metric_previous_values[metric]
            self.metric_previous_values[metric] = int(metric_value)

        return metric_values

    def get_labels_for_pod(self, pod_id):
        """
        Get labels on the pause container for the tagging purpose later on
        :param pod_id:
        :return: k-v pair of labels
        """
        containers = self.docker_client.containers.list(filters={'label': 'io.kubernetes.pod.uid=%s' % pod_id})
        for c in containers:  # look for pause container because it has all the labels
            if 'Config' in c.attrs:
                if 'Entrypoint' in c.attrs['Config'] and c.attrs['Config']['Entrypoint'][0] == '/pause':
                    return self.extract_relevant_labels(c.attrs['Config']['Labels'])
        return dict()  # empty dict if not found

    def extract_relevant_labels(self, labels):
        """
        Extracts only relevant labels from all the labels on the docker container
        :param labels:
        :return: k-v pair of relevant labels
        """
        extracted = [(label, labels[label]) for label in self.relevant_labels if label in labels]
        return dict(extracted)

Hello @tsunny, thank you for this suggestion, it would be a nice addition indeed!
You can submit a PR in integrations-extras (https://github.com/DataDog/integrations-extras) so it would be available as a third-party integration and we would review the code there.
You use our dev tool to create the integrations necessary files https://datadoghq.dev/integrations-core/ddev/cli/#ddev-create


implement a global deadline when running testcode in the local arangosh

this enables us to have a timeout for testfiles that we eval' into the testing.js arangosh by runInLocalArangosh.

The results of this can be observed by specifying a small timeout:
./scripts/unittest shell_client --oneTestTimeout 10
2020-02-20T12:26:59.397Z [============] runInLocalArangosh: Trying tests/js/common/shell/aql-index-usage-rocksdb.js ...

2020-02-20T12:26:59.400Z [------------] 1 tests from IndexUsageSuite (setUpAll: 0ms)
2020-02-20T12:26:59.400Z [ RUN        ] testIndexUsage
2020-02-20T12:27:09.404Z [   FAILED   ] testIndexUsage (setUp: 101ms, test: 9902ms, tearDown: 1ms)
ArangoError: Timeout for remote!
    at ArangoCollection.getIndexes.ArangoCollection.indexes (/home/willi/src/devel4/js/client/modules/@arangodb/arango-collection.js:671:50)
    at Object.testIndexUsage (tests/js/common/shell/aql-index-usage-rocksdb.js:101:34)
    at Object.run (/home/willi/src/devel4/js/common/modules/jsunity/jsunity.js:549:27)
    at Object.Run [as run] (/home/willi/src/devel4/js/common/modules/jsunity.js:275:24)
    at tests/js/common/shell/aql-index-usage-rocksdb.js:128:9
    at tests/js/common/shell/aql-index-usage-rocksdb.js:131:3
    at RunTest (/home/willi/src/devel4/js/common/modules/jsunity.js:389:12)
    at testFunc (eval at runInLocalArangosh (/home/willi/src/devel4/js/client/modules/@arangodb/test-utils.js:889:3), <anonymous>:4:9)
    at runInLocalArangosh (/home/willi/src/devel4/js/client/modules/@arangodb/test-utils.js:893:18)
    at Object.performTests (/home/willi/src/devel4/js/client/modules/@arangodb/test-utils.js:233:21) - test failed
ArangoError: Timeout for remote!
    at ArangoCollection.drop (/home/willi/src/devel4/js/client/modules/@arangodb/arango-collection.js:510:48)
    at Proxy.ArangoDatabase._drop (/home/willi/src/devel4/js/client/modules/@arangodb/arango-database.js:442:29)
    at Object.tearDown (tests/js/common/shell/aql-index-usage-rocksdb.js:57:10)
    at /home/willi/src/devel4/js/common/modules/jsunity/jsunity.js:483:16
    at Object.run (/home/willi/src/devel4/js/common/modules/jsunity/jsunity.js:554:19)
    at Object.Run [as run] (/home/willi/src/devel4/js/common/modules/jsunity.js:275:24)
    at tests/js/common/shell/aql-index-usage-rocksdb.js:128:9
    at tests/js/common/shell/aql-index-usage-rocksdb.js:131:3
    at RunTest (/home/willi/src/devel4/js/common/modules/jsunity.js:389:12)
    at testFunc (eval at runInLocalArangosh (/home/willi/src/devel4/js/client/modules/@arangodb/test-utils.js:889:3), <anonymous>:4:9) - tearDown failed
2020-02-20T12:27:09.404Z [------------] 1 tests from IndexUsageSuite ran (tearDownAll: 1ms)
2020-02-20T12:27:09.404Z [   PASSED   ] 0 tests.
2020-02-20T12:27:09.404Z [   FAILED   ] 1 tests.
2020-02-20T12:27:09.404Z [============] Ran: 1 tests (0 passed, 1 failed) (10004ms total)

oops! Skipping remaining tests, server is gone.
Thu Feb 20 2020 13:27:09 GMT+0100 (Central European Standard Time) Shutting down...

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8668/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8728/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8748/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8843/

Rename "deadline"

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8874/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8900/

Since each of you @ObiWahn @KVS85 @jsteemann  raised concerns about this functionality also being available there in the server, I modified it to be available in clients only.
http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/8911/

You use setExecutionDeadlineInMS only in the function that is available to the client. But it is not obvious at all that people are not allowed to use this function in the server. You need at least some comment or better move the code altogether to arangosh.

Again - we may fix the usage of chrono once we do this for js_download. its out of scope for this PR.

Just use chrono for your new code and cast to others as required.

I (we) did, on all places that already use it. js_download will most probably completely be replaced soon to get rid of that http implementation, at that time the implementation can simply be erased.

tests green.

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/9010/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/9012/

tests all blue.

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/9022/

http://jenkins01.arangodb.biz:8080/view/PR/job/arangodb-matrix-pr/9026/


Wrench in parent topics creates the impression there are no child topics

Issue Title
Wrench in parent topics creates the impression there are no child topics
General
Because child nodes are not expanded by default, a wrench, which indicates that a topic has not been written but consists of boilerplate text only, creates the impression that child topics are similarly unwritten.
Issues with Existing Topics
This applies to the following topics:

docs/csharp/getting-started/index.md (C# Getting Started
docs/csharp/concepts.md C# Concepts
docs/csharp/methods-lambda-expressions.md Methods and Lambda Expressions. I wonder if this just could be eliminated and the Methods and Lambda Expressions topics moved up one level in the TOC.

can we close this @rpetrusha?

@mairaw Yes. I'm closing it now.


It does not pickup typescript in monorepo environment w/ pnpm

Description
I have a monorepo managed with pnpm, where typescript dependency lives in the root package.json file.
Now, openapi-ts does not see typescript from root module and requires typescript that is installed locally.
> web-renter-portal@ gen:api-local /Users/johndoe/Projects/portal/apps/web-renter-portal
> openapi-ts --input http://<IP_ADDRESS>:5000/swagger/swagger.json --output app/api --client fetch --useOptions

🔥 Unexpected error occurred. Log saved to /Users/johndoe/Projects/portal/apps/web-renter-portal/openapi-ts-error-1715779812572.log
🔥 Unexpected error occurred. 🚫 dependency missing - TypeScript must be installed

OpenAPI specification (optional)
No response
Configuration
No response
System information (optional)
No response

Please make sure you're on the latest version :)

@mrlubos oh, I just had to be on the latest version :-D
Thanks and apologies for wasting your time.

No worries @vladmiller, thank you for using this package!


Example of asymmetric

I see only private_key and public_key text file. Can I have a example on use of that private key and public key?

Hi @himalacharya, currently in the documentation doesn't have an example to use an asymmetric algorithm, I'm working on to update the documentation. If documentation is ready I will mention you again thanks 🙏

You can check the documentation here Link 😄


Ansible - API key decode

We are getting an error very regularly where jobs running via Ansible are not able to decode the API Key.
"Creating PTR Record for <IP_ADDRESS>.in-addr.arpa..",
"Error. Unable to decode the API Key. Please set the Global API Key again using: Set-ibPSConfiguration -CSPAPIKey  or create a new connection profile.\r\r",
"If you have recently upgraded from a version older than v<IP_ADDRESS>, you will need to update your API Key.\r\r",
"See the following link for more information: ",
"https://ibps.readthedocs.io/en/latest/#authentication-api-key\r\r",
Logging in as the user, everything seems to be fine.
However jobs ran as the user on that script server get the above error whether we set the api key and its stored as a environment variable or if its setup as a connection profile.
Any ideas?

Hey,
Can you try using the latest dev release, and pass the API Key as an environment variable as described here;
https://ibps.readthedocs.io/en/dev/#environment-variable

Workaround using Environment Variables available in latest release: v<IP_ADDRESS>


[Event Request] codeunit 7314 "Warehouse Availability Mgt." - OnCalcQtyOnSpecialBinsOnLocation

We'd like to add extra special bins to the SpecialBins list in codeunit 7314 "Warehouse Availability Mgt.", function CalcQtyOnSpecialBinsOnLocation.
Can you please add integration event with write access to SpecialBins?
procedure CalcQtyOnSpecialBinsOnLocation(LocationCode: Code[10]; ItemNo: Code[20]; VariantCode: Code[10]; WhseItemTrackingSetup: Record "Item Tracking Setup"; var TempBinContentBufferExcluded: Record "Bin Content Buffer" temporary) QtyOnSpecialBins: Decimal
    var
        Location: Record Location;
        SpecialBins: List of [Code[20]];
        SpecialBin: Code[20];
    begin
        Location.Get(LocationCode);

        if Location."To-Assembly Bin Code" <> '' then
            SpecialBins.Add(Location."To-Assembly Bin Code");
        if (Location."Open Shop Floor Bin Code" <> '') and not SpecialBins.Contains(Location."Open Shop Floor Bin Code") then
            SpecialBins.Add(Location."Open Shop Floor Bin Code");
        if (Location."To-Production Bin Code" <> '') and not SpecialBins.Contains(Location."To-Production Bin Code") then
            SpecialBins.Add(Location."To-Production Bin Code");

        OnCalcQtyOnSpecialBinsOnLocation(Location, SpecialBins); // <<< NEW EVENT

        foreach SpecialBin in SpecialBins do begin
            TempBinContentBufferExcluded.SetRange("Location Code", LocationCode);
            TempBinContentBufferExcluded.SetRange("Bin Code", SpecialBin);
            if TempBinContentBufferExcluded.IsEmpty() then
                QtyOnSpecialBins +=
                    CalcQtyOnBin(LocationCode, SpecialBin, ItemNo, VariantCode, WhseItemTrackingSetup);
        end;
    end;

Suggested event
 [IntegrationEvent(false, false)]
    local procedure OnCalcQtyOnSpecialBinsOnLocation(Location: Record Location; var SpecialBins: List of [Code[20]])
    begin
    end;

PS: Or move the below code to a separate GetSpecialBins function that fills the list with Special bins with an OnAfterGetSpecialBins event?
 if Location."To-Assembly Bin Code" <> '' then
            SpecialBins.Add(Location."To-Assembly Bin Code");
        if (Location."Open Shop Floor Bin Code" <> '') and not SpecialBins.Contains(Location."Open Shop Floor Bin Code") then
            SpecialBins.Add(Location."Open Shop Floor Bin Code");
        if (Location."To-Production Bin Code" <> '') and not SpecialBins.Contains(Location."To-Production Bin Code") then
            SpecialBins.Add(Location."To-Production Bin Code");

        OnAfterGetSpecialBins(Location, SpecialBins); // <<< NEW EVENT

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 10868.