Memory leak on Interceptor

Report
I think we have a memory leak.
Here is the memory usage over 2 days:

The memory drops is the pod getting OOM killed. At first we thought 64mb is too little, so we decided to experiment higher values in the last 24hrs, but the symptom persisted:

Expected Behavior
the pods should release memory and as traffic fluctuates down and not get OOM killed
Actual Behavior
Memory usage never decreases
Steps to Reproduce the Problem

Install version 0.8
Put a lot of load on apis controlled by this guy

Logs from KEDA HTTP operator
example

HTTP Add-on Version
0.8.0
Kubernetes Version
< 1.28
Platform
Microsoft Azure
Anything else?
AKS v 1.27.9

@JorTurFer / @wozniakjan do you have bandwidth to check this one?

thank you for reporting this, from the graphs this indeed looks like a memory leak, I can investigate later this week.

after a quick investigation, this might actually be related to HTTP connections overhead, GC and request caching rather than an actual memory leak. I am able to reproduce quick memory growth with concurrent requests. When idle on a tiny cluster with a single HTTPScaledObject, the interceptor uses around 21Mi
$ kubectl top pods -nkeda --selector app.kubernetes.io/component=interceptor
NAME                                            CPU(cores)   MEMORY(bytes)
keda-add-ons-http-interceptor-6f6d78bd9-rkn2r   4m           21Mi

with requests sent sequentially, the memory usage barely increases and then goes down to idle levels. With simple benchmark using wrk with 100 concurrent threads and 100 connections
$ wrk -t100 -c100 -d30s -H "Host: myhost2.com" http://<IP_ADDRESS>

the memory usage does increase, peaking at 54Mi:
$ kubectl top pods -nkeda --selector app.kubernetes.io/component=interceptor
NAME                                            CPU(cores)   MEMORY(bytes)
keda-add-ons-http-interceptor-6f6d78bd9-rkn2r   494m         54Mi

but after wrk benchmark terminates, the CPU usage drops down quickly and while the memory usage decreases rather slowly, both eventually reach similar levels to the initial idle. With 100 threads and 500 connections, interceptor OOMKills pretty fast with a default limit of 64Mi, after bumping the memory limit to 1Gi, the memory again seems well flexible - growing as well as shrinking with benchmark executions.
Attached is also trace output from net/http/pprof during the execution of the wrk benchmark. The memory growth is apparent but so is also the memory decrease. There doesn't appear to be any place in code where the memory just grows. I will perform some longer running benchmarks with profiling traces to try to spot something more sneaky.
trace-wrk.txt

I may have noticed something, with higher error rate in the benchmark, the memory consumption doesn't go down. I just bumped the memory limit for interceptor to 1Gi and executed benchmark that may have been too much, 5000 connections in 200 threads for 10 minutes
$ wrk -t200 -c5000 -d10m -H "Host: myhost2.com" http://<IP_ADDRESS>
Running 10m test @ http://<IP_ADDRESS>
  200 threads and 5000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   688.19ms  506.07ms   2.00s    69.41%
    Req/Sec     6.19      5.99   101.00     83.53%
  351661 requests in 10.00m, 277.64MB read
  Socket errors: connect 0, read 0, write 0, timeout 323052
  Non-2xx or 3xx responses: 16769
Requests/sec:    586.00
Transfer/sec:    473.75KB

out of ~350000 requests, 91% timeouts and 4.7% errors. The memory usage is at 531Mi which is close to its ~600Mi peak during the benchmark and it doesn't seem to go down.

@wozniakjan my current number is around 15k requests every minute. I currently have 3 interceptor pods, at 640 MB each... in about one hour and half that number is expected to peak at 25k... during the late night, between 2am-5am, the number drops to 5k...

@Leonardo-Ferreira I played around with benchmarks a bit more and afaict interceptor seems to be very CPU intensive, for wrk simulated load that would match the request rate of 25k/min I am able to saturate 12 CPU cores easily. With this amount of CPU, it's able to maintain a relatively low error rate 0.42%.
With a low error rate, the memory usage appears to go down to idle levels eventually, but when the error rate is high, even when later the interceptor is off traffic, memory usage doesn't seem to go down.
Go's pprof heap analysis doesn't point to any code that would accumulate memory and not release it, but also heap as counted by pprof appears to be just a fraction of memory used compared to cgroup accounting. The pprof recorded 44MB while cgroup show 318MB used.

@wozniakjan after looking deeply at my data, I could not correlate the memory usage/increase-rate to the number of errors. Luckily (or not) we had an issue this morning where a dependency was flaky so there was a significant spike in errors related to timeouts. on the memory consumption graph there was no significant change during the 1 hour long event...

well, that's weird... our pods have a limit of 0.7

good to hear that, perhaps my artificial benchmark setup just ends up being CPU-intensive.
my current hypothesis I am trying to either prove or disprove is that maybe when the interceptor itself fails to process the requests and it has a higher error rate, maybe then it leaves some resources dangling.

would that output a specific log message that I could query for?

should we log some extra info and release a 0.8.1 version? unfortunately my security dpto does not allow me to use "unofficial" images in production

I'm still in the process of trying to figure out the root cause but as soon as I have some valid hypothesis to test, I can distribute a build for your testing

any news @wozniakjan, can I help somehow?

I didn't get very far and now I had to put it on hold temporarily, but I will get back to this soon

hey @wozniakjan, I'd like to contribute here. would you be willing to connect for like 30-45min so you can "boost" me? this way I could make significant contributions faster

sure, you can ping me on KEDA slack https://keda.sh/community/
https://kubernetes.slack.com/team/UF5R8HHB7

@zroubalik reopen?


Add ArchReports and IssueEntry object

Obsolete Report object
Fix soft failed by using new result_softfailed class string

I merged "Fix soft failed: Use new 'result_softfailed' str"
The rest is a mixture of refactoring and adding more information to the reports which is what the test verbosity is for. The terse report should be "terse" and should not include the URLs, it's not anymore


Pre realease 0.3.1

We could start using some automation for release notes. We could use this as standard: https://www.conventionalcommits.org/en/v1.0.0/


Port to mycropython

Do you think that it could be possible to port the script to a ESP32 C3 micropyton environment?
Thanks

Hey, unfortunately, I have no experience with ESP32 and don't have the hardware.


Make the slot duration part of RunNode

When starting a node, the nodeSlotDuration method can be used to obtain the
slot duration.  For example, for real Byron blocks, it will extract the slot duration from the protocol parameters in the Genesis config. For mock blocks, it will just use a hard coded slot duration.

I don't think this approach is quite right. The slot length should be determined by the consensus protocol in question, not externally. Moreover, slot lengths will not be a fixed number. So, for example, @ruhatch 's hard fork protocol combinator should be able to say "the slot length is x up to the hard fork, and y after".

I don't think this approach is quite right. The slot length should be determined by the consensus protocol in question, not externally. Moreover, slot lengths will not be a fixed number. So, for example, @ruhatch 's hard fork protocol combinator should be able to say "the slot length is x up to the hard fork, and y after".

So how should we proceed? Add protocolSlotLength :: NodeConfig p -> SlotLength to OuroborosTag and use that in cardano-node? When we change the SlotLength to be variable, we'll just update it (we should not let that block the clean-up of cardano-node).
Another thought: instead of letting cardano-node set up the BlockchainTime using the SystemStart and SlotLength, we could just let consensus provide the BlockchainTime. For example, we could add  nodeBlockchainTime :: Proxy blk -> NodeConfig (BlockProtocol blk) -> ResourceRegistry m -> BlockchainTime m to RunNode. Actually, my goal is to keep even that encapsulated in consensus code, I'd like to just provide a function (taking some parameters) that simply runs the node (opening the ChainDB, starting the NodeKernel, setting up the network stack).

So how should we proceed? Add protocolSlotLength :: NodeConfig p -> SlotLength to OuroborosTag and use that in cardano-node? When we change the SlotLength to be variable, we'll just update it (we should not let that block the clean-up of cardano-node).

Yes, I think that makes sense.

As for RunNode, I would propose slimming that down as much as possible, and reserve it for stuff that's actually type level and then instead provide something like
data RunNodeArgs blk = RunNodeArgs {
     runNodeStartTime :: ...
  , runNodeWhatever :: ...
   }

and then provide a function that has a type level constriant RunNode blk as well as a value level RunNodeArgs blk. Doesn't seem useful tohave all of this in a type class.

As for RunNode, I would propose slimming that down as much as possible, and reserve it for stuff that's actually type level and then instead provide something like
data RunNodeArgs blk = RunNodeArgs {
     runNodeStartTime :: ...
  , runNodeWhatever :: ...
   }

and then provide a function that has a type level constriant RunNode blk as well as a value level RunNodeArgs blk. Doesn't seem useful tohave all of this in a type class.

Who will create these RunNodeArgs? cardano-node? Or consensus?
(runNodeStartTimeshould be extracted from the genesis config in case of Byron blocks)

I guess consensus should; providing functions that construct RunNodeArgs for the Byron era, and then another one for the Shelley era (as well as others used internally for mock protocols).

See #953 (WIP still).


First time connection displayed in red

Although it is connected

IsConnected = eventArgs.IsConnected && _interactiveWorkflow.ActiveWindow != null;
REPL is not up yet

Requested by #2736

Yeah but it doesn't change b/c (a) there is no UpdateActiveConnection in  ActiveWindowChanged and (b)  _sessionProvider.IsConnected is still false when REPL becomes active and CM is not listening to Connected

 ActiveWindowChanged is raised only after RSession is connected:
            if (!RSessions.HasBroker) {
                var connectedToBroker = await Connections.TryConnectToPreviouslyUsedAsync();
                if (!connectedToBroker) {
                    Connections.GetOrCreateVisualComponent().Container.Show(focus:false, immediate:false);
                }
            }

            await interactiveWindow.InitializeAsync();

            ActiveWindow = window;
            ActiveWindow.Container.UpdateCommandStatus(true);
            _visualComponentTcs.SetResult(ActiveWindow);
            ActiveWindowChanged?.Invoke(this, new ActiveWindowChangedEventArgs(window));

_sessionProvider.IsConnected is false

That is a regression from #2793 - event is raised but property isn't updated

Because host info really should not have been used for the connectivity and broker state tracking in the first place.


Add all strings to resources

@HugoGresse Should error_general really be quoted?

@mueller-ma I don't think so but it should prevent any issue if the error number does not exist.

@HugoGresse It's easier for me, when you translate it. Idk how to update my pr. retry = "Wiederholen"

sure np. thanks


docs(devops): add instructions for additional workflow

Closing in favor of this location for scripts and DevOps


the users balance become negative when admin function  presses function in last trade menu

the administration function that looks up the " last trade" function  has little "x" side of the reported last trade. When you press this the trade is actually deleted from the data base leading to negative and screwed up user wallet balance as if the trades they made never happened.
This leads to negative number in users wallet .
How do you fix the balance. I tried opening up the
\opentrade\server\database\sqlite.db
and
\opentrade\accountsserver\database\sqlite_accounts.db
with Sqlite browser and editing them eventually all the data is written back again when the opentrade connects with the Coin wallet.
IS there another data base that information is saved for users
" last trade" function   causes problems by deleting some data leading to negative numbers in users Wallet.
the  "x" should not delete any records. This caused a lot of problems.

Do not use this function if you are not exactly sure what you are doing.
Do not edit database with external instruments  if you are not exactly sure what you are doing.
The negative balance should be fixed from admin panel -> Check Balance tab.

Do not use this function if you are not exactly sure what you are doing.
Do not edit database with external instruments if you are not exactly sure what you are doing.
The negative balance should be fixed from admin panel -> Check Balance tab.

Please tell us  what is admin " last trade" function has little "x" side function is for and what is meant for?. this is why I am here to learn about this program and contribute to the cryptocurrency best I can. Others people on this forum will benefit too as it goes forward.
Thanks for your help
I thought it would allow the admin scroll through the last trade but it actually seems to delete the,
The problem occurred before external browser for sqlite  used. It occurs when " last trade" function has little "x" side used
I already used  " check balance " and pressing on "fix" tab did not work.
It is essential that exchange  should have function to adjust balance.  It is necessary when you are hacked or if you need to refund a user


low borg charge should affect more

so that cyborgs can't walk when discharged
and can only beep to indicate that they has no charge

they already have a lowered max speed when out of battery. I like disabling speech as a form of it, too. Maybe, in the future, we can have more tools and mechanics tied in to battery (once the system is more robust)


My game is a spin-off of a Sokoban game in which a fox has to push chests into portals in order to escape its enemy. The goal is to get every chest to a portal.

Example: Pushing boxes to the goal. (from Sokoban Plus)

How to play your game?
To play the game, you can use WASD to move, and J to restart the level.

Example  : Press WASD to move, J to restart and K to toggle trails, Get A boxes (cyan) to A goals (green), Get B boxes (magenta) to B goals (red), Get normal boxes (gray) to either goal. (from Sokoban plus)

Code
Check off the items that are true.

[x] The game was made using the Sprig editor.
[x] The game is placed in the in the /games directory.
[x] The code is significantly different from all other games in the Sprig gallery (except for games labeled "demo").
[x] The game runs without errors.
[x] The name of the file/game contains only alphanumeric characters, -s, or _s.
[x] The game name is not the same as the others from gallery

Image (If an image is used)

[x] The image is in the /games/img directory.
[x] The name of the image matches the name of your file. Example: sokoban_plus.js -> sokoban_plus.png.

Thanks for PR!

add more mechanisms to make your code more different from others:)

It seems you need to make more changes before you're ready to get your Sprig. I'm going to close this PR for now. Feel free to reopen if you want to continue working on your game.


cannot enable "Support pages and views" for WebAssembly client projects

Description
[Enter feedback here]
You should highlight that you cannot enable "Support pages and views" for WebAssembly client projects. If you do, you get the painful error, "no runtime pack for Microsoft.AspNetCore.App available for the specified RuntimeIdentifier 'browser-wasm'"
I highlighted the changes made and a fix here - https://github.com/dotnet/aspnetcore/issues/53671#issuecomment-2081461812
Page URL
https://learn.microsoft.com/en-us/aspnet/core/blazor/components/class-libraries?view=aspnetcore-8.0&tabs=visual-studio
Content source URL
https://github.com/dotnet/AspNetCore.Docs/blob/main/aspnetcore/blazor/components/class-libraries.md
Document ID
a92c88d3-6ee8-1e69-f4c3-49eb82909746
Article author
@guardrex

Thanks @ghhv! ... I'll get a PR up shortly to address this.


Catch unwinding panics across FFI boundary

Currently, generated interface implementation (including callback consumption) does not catch unwinding panics. By default, when a panic occurs in user code, the thread starts unwinding all the way down to system dlls. Since unwinding across FFI boundary is undefined behavior, user code invocations should be wrapped in catch_unwind.

Yeah, I'm still trying to figure out how best to handle this. On Windows, we generally have an abort/terminate function that immediately terminates the process (e.g. TerminateProcess) without unwinding in order to collect reliable crash dumps. This is distinct from "exceptions" that may unwind the stack, and we certainly don't want to allow exceptions to unwind as that is not reliable across ABI boundaries in C++ either. So ideally, we could ensure that panic does not unwind but if I don't know whether that is practical.
And if catch_unwind is used, the question is what to do with the resulting error. I don't want it to propagate but neither do I want to simply swallow the error and lose that context.

Do not terminate accounts have been engaged in illegal activities and police reports filed they are panicking

I don't want it to propagate

May I know the reason not to do so?

Because the ABI semantics don't allow for it. Errors must be reported via return codes. Unwinding is also not compatible across compilers and languages.

Can we mimic the behavior when an exception is thrown in cppwinrt, like reporting the error via RoOriginateLanguageException?

FWIW there is an RFC to reduce undefined behavior when unwinding across FFI boundaries, but it looks to be fairly static in the last year or so: https://github.com/rust-lang/rfcs/blob/master/text/2945-c-unwind-abi.md

Just looking again at this old issue. I think I'm going to leave this up to the implementor. I don't see what the implement macro can do beyond simply aborting in such cases, in which case it would be more efficient to do that declaratively:
https://doc.rust-lang.org/book/ch09-01-unrecoverable-errors-with-panic.html


Which model is being run for img2dataset

Does it useclip-l-14-336px ?

Img2dataset does not run clip
On Mon, Jun 26, 2023, 22:01 junwang-wish @.***> wrote:

Does it useclip-l-14-336px ?
—
Reply to this email directly, view it on GitHub
https://github.com/rom1504/clip-retrieval/issues/289, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAR437RILQC6FTQ6WQTF4NLXNHTDFANCNFSM6AAAAAAZUUFGOI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>


Shortcut key for opening folders does not work when executed immediately upon launch

VSCode Version: 1.38.1
OS Version: Linux x64, Windows 10

Steps to Reproduce:

Launch Visual Studio Code
After VSCode completely loads, launch a new VSCode window using the shortcut key (Ctrl + Shift + N)
Immediately (within one second) after the new VSCode window is rendered, attempt to open a folder using the shortcut key (Ctrl + K, Ctrl + O)
A new dialog will appear, but the dialog will only provide the user the option to open files, not folders
However, if the user waits for at least a few seconds after the VSCode window has rendered before attempting to open the folder using the shortcut key, then the presented dialog will provide the user with the expected option to open folders

Does this issue occur when all extensions are disabled?: Yes

@eddylu94 I cannot reproduce anymore:

Please let me know if you can still reproduce.

Hi @alexdima, thanks for following up with this issue. Since this ticket was created, I have actually changed platforms and no longer utilize an environment in which I can test/reproduce this issue. Therefore, if no other tickets have been filed for this issue, and since there have been no upvotes for this ticket, then I believe it is reasonable to close this ticket.


Create CODEOWNERS

Description
I'm setting up a CODEOWNERS file with people that expressed interest and also trying to get @aaronb97 and @bvarberg involved even though they didn't express interest.
CODEOWNERS

Code owners are automatically requested for review when someone opens a pull request that modifies code that they own. Code owners are not automatically requested to review draft pull requests. For more information about draft pull requests, see "About pull requests." When you mark a draft pull request as ready for review, code owners are automatically notified. If you convert a pull request to a draft, people who are already subscribed to notifications are not automatically unsubscribed. For more information, see "Changing the stage of a pull request."
When someone with admin or owner permissions has enabled required reviews, they also can optionally require approval from a code owner before the author can merge a pull request in the repository. For more information, see "About protected branches."
If a file has a code owner, you can see who the code owner is before you open a pull request. In the repository, you can browse to the file and hover over .

@luiz-chagas let's add @blaketarter to the list as well. He's been involved in these efforts for years now.

💭 we probably just need to be aware that anyone who uses this template as a base will need to adjust the codeowners right off the bat

@aaronb97 ah, yeah, this might be why we haven't set them up for this project. 🤔

I wish there was a way to control what files get copied over when the Repo is used as a template..

💭 we probably just need to be aware that anyone who uses this template as a base will need to adjust the codeowners right off the bat

True, I'll leave this closed until there's a "template project ignore" kind of thing.


clusterAgent is missing resources and using latest tag

Two items that should be addressed before clusterAgent is released:

clusterAgent image is configured to use latest tag; I assume this should be pinned to a release
clusterAgent containers do not specify any requests or limits for resources - should it expand resources from values?

bumping this. Release 6.6.0 still points to latest rather than 6.6.0.

Thanks for the callout -- we've fixed this for upcoming versions (v6.7 and beyond), and I'll modify the current v6.6 charts to include this as well.


calico CNI command never terminates, which causes a dockershim server handler to stuck forever

To give some context it's based on already closed issue:
https://github.com/projectcalico/calico/issues/1109
If this is redundant feel free to close it.
Please take a look at comment which is crucial:
https://github.com/kubernetes/kubernetes/issues/45419#issuecomment-405168344
Summary:
The bug is caused by a calico CNI command that never terminates, which causes a dockershim server handler to stuck forever. As a result, the RPC calls PodSandboxStatus() for a bad pod always timeout, thus makes the PLEG to be unhealthy.
Expected Behavior
Calico CNI command timeout and terminates with success or failure.
Current Behavior
Calico CNI command that never terminates.
This bug could be probably also fixed in CNI (or kubelet): https://github.com/containernetworking/cni/pull/568
But fixing it directly in calico can increase resiliency of the system.

Yeah, sounds like we should be handling this better.
Do we know which call is hanging in this specific instance?

I've gathered some corresponding logs. Issue is caused by removing workload opengraph-7cdff79dbb-rsb4g in this case.
kubelet:
I0821 10:33:59.709000   23712 kubelet.go:1853] SyncLoop (DELETE, "api"): "opengraph-7cdff79dbb-rsb4g_prod(13dd6d43-9fc1-11e8-b4ea-525400215c79)"
[...]
E0821 10:36:03.198396   23712 remote_runtime.go:115] StopPodSandbox "1a8202c0a88fe079be790d10b4840444a92a7ca676130b43903aec0e684fd22f" from runtime service failed: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0821 10:36:03.198473   23712 kuberuntime_manager.go:799] Failed to stop sandbox {"docker" "1a8202c0a88fe079be790d10b4840444a92a7ca676130b43903aec0e684fd22f"}
E0821 10:36:03.198523   23712 kubelet.go:1512] error killing pod: failed to "KillPodSandbox" for "13dd6d43-9fc1-11e8-b4ea-525400215c79" with KillPodSandboxError: "rpc error: code = DeadlineExceeded desc = context deadline exceeded"
E0821 10:36:03.198573   23712 pod_workers.go:186] Error syncing pod 13dd6d43-9fc1-11e8-b4ea-525400215c79 ("opengraph-7cdff79dbb-rsb4g_prod(13dd6d43-9fc1-11e8-b4ea-525400215c79)"), skipping: error killing pod: failed to "KillPodSandbox" for "13dd6d43-9fc1-11e8-b4ea-525400215c79" with KillPodSandboxError: "rpc error: code = DeadlineExceeded desc = context deadline exceeded"
E0821 10:36:03.344535   23712 kuberuntime_manager.go:859] PodSandboxStatus of sandbox "1a8202c0a88fe079be790d10b4840444a92a7ca676130b43903aec0e684fd22f" for pod "opengraph-7cdff79dbb-rsb4g_prod(13dd6d43-9fc1-11e8-b4ea-525400215c79)" error: rpc error: code = DeadlineExceeded desc = context deadline exceeded
E0821 10:36:03.344611   23712 generic.go:241] PLEG: Ignoring events for pod opengraph-7cdff79dbb-rsb4g/prod: rpc error: code = DeadlineExceeded desc = context deadline exceeded

Full kubelet log:
https://gist.github.com/r0bj/f698f26e0b83f674c087d3cd8f402c40
calico-node:
2018-08-21 10:34:03.313 [INFO][142] calc_graph.go 359: Local endpoint deleted id=WorkloadEndpoint(node=k8s-worker-r22, orchestrator=k8s, workload=prod/opengraph-7cdff79dbb-rsb4g, name=eth0)

Full calico-node log:
https://gist.github.com/r0bj/70301cc32ab2444c0f9127b1feaab620
dmesg:
[Tue Aug 21 11:51:05 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:51:15 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:51:25 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:51:35 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:51:45 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:51:55 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:52:05 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:52:16 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1
[Tue Aug 21 11:52:26 2018] unregister_netdevice: waiting for eth0 to become free. Usage count = 1

I hope it'll be useful.


AttributeError: 'dict' object has no attribute 'get_epoch_log_metrics'

I have created a lightning module which is working fine for single validation dataset but throws following error while using multiple validation dataset:
self._log_on_evaluation_epoch_end_metrics(epoch_logs)
  File "/home/ubuntu/anaconda3/envs/pytorch_latest_p36/lib/python3.6/site-packages/pytorch_lightning/trainer/connectors/logger_connector.py", line 186, in _log_on_evaluation_epoch_end_metrics
    logger_metrics = reduced_epoch_metrics.get_epoch_log_metrics()
AttributeError: 'dict' object has no attribute 'get_epoch_log_metrics'

Here's my validation_step method. To support single as well as multiple validation dataset I have kept dataset_idx = 0
    def validation_step(self, batch, batch_idx, dataset_idx=0):
        qa_pairs, labels = batch
        # labels = labels.type(torch.LongTensor)
        labels = labels.long()#(torch.LongTensor)
        # run the model for the inputs
        outputs = self(qa_pairs)
        # compute the loss
        if self.loss.lower() == "cross_entropy":
            loss = F.cross_entropy(outputs, labels)

        self.valid_acc(outputs, labels)
        self.log('val_loss', loss, on_step=True)
        self.log('val_acc', self.valid_acc, on_step=True, on_epoch=True, prog_bar=True)

Looking at the error msgs, I commented the logging part, which seems to get rid of the error.
system:
pytorch: 1.6
ubuntu 18.04
pytorch-lightning: 1.0.3
StackOverflow Post

related #3994?


[ENH] - Make idle culler settings easily configurable and documented how to change

Feature description
Currently much of the idle culler is hard coded. @rsignell-usgs brought this up as an issue that he was concerned about. The current timeout is too short in some cases.
Value and/or benefit
The default idle timeout does not work for everyone.
Anything else?
No response

Hi @costrouc, do they want a config per user structure, or they are happy to get it set in the qhub-config?

@viniciusdc and @costrouc , we would be happy to set this in the qhub-config.

Folks, what would it take to enable this?   This is the number #1 complaint I've heard from ESIP Qhub users.  Even if it wasn't configurable and just made longer by qhub devs, that would be wonderful.  Right now it must be 5 minutes, right?   It would be great if dask clusters spun down in 30 min, and notebooks spun down in 90 min or 3 hours.

Hi @rsignell-usgs, I will make sure this issue is prioritized for our next sprint (which starts next week). I can't promise it will be configurable from the qhub-config.yaml but I will work with the team to come up with a workable solution asap. Thanks again for the reminder!!

@iameskild , I remember you showed me how to (temporarily) override the short culler settings by connecting to some pod and editing a config file, right?     After the upgrade from 0.4.3 to 0.4.4, the users are screaming again about the too-short timeout for their servers.

Hey @rsignell-usgs, for now,  you can manually edit the etc-jupyter configmap if you want to make changes to the timeout settings.
Although I still have to circle back to this when I have more time but as a quick update, I was looking into using Terraform's templatefile to make these values more easily configurable.

This can also be achieved using overrides on the jupyterhub configuration to change the idle-culling variable values. Right now, the values that can be changed are those here
There are some values that comes from the idle-culler extension that as of now, only the above method can be used

To change these, I can use k9s to ssh into the hub-** pod and then just edit them?

@rsignell-usgs yep, just edit the file. You may need to kill the hub pod for the changes to take effect.

What is the filename once I've ssh'ed into the hub pod?

Here's the recipe:

in k9s, type "configmap"
use arrow keys to highlight the etc-jupyter configmap
hit the e key to edit
still in k9s, type "pod"
use arrow keys to highlight the pod that starts with hub-xx
delete the pod (it will regenerate in just a few seconds)

Just for the record, I set everything to 30 minutes:

    # The interval (in seconds) on which to check for terminals exceeding the
    # inactive timeout value.
    c.TerminalManager.cull_interval = 30 * 60

    # cull_idle_timeout: timeout (in seconds) after which an idle kernel is
    # considered ready to be culled
    c.MappingKernelManager.cull_idle_timeout = 30 * 60

    # cull_interval: the interval (in seconds) on which to check for idle
    # kernels exceeding the cull timeout value
    c.MappingKernelManager.cull_interval = 30 * 60

    # cull_connected: whether to consider culling kernels which have one
    # or more connections
    c.MappingKernelManager.cull_connected = True

    # cull_busy: whether to consider culling kernels which are currently
    # busy running some code
    c.MappingKernelManager.cull_busy = False

    # Shut down the server after N seconds with no kernels or terminals
    # running and no activity.
    c.NotebookApp.shutdown_no_activity_timeout = 30 * 60


Export the bootstrap wizard commands as flux-exec options

As we agreed, exporting the options as flux-exec options is too much hassle for now.
So, the options will be exported as part of the result object which will also contain info on the Git provider.
I will also add constants for all bootstrap wizard option keys.


Text Editor theme not all Basic Types are affected by colouring

In Text Edit settings there is option to change color for Basic Types. It works for Vector, Matrix, Transform, Plane, Quat, InputEvent and Color types.
Other base types like String, NodePath, IntArray and so on are not affected.
Well also int/float numeric values are not affected but they probably need their own color setting.

Eh, I know remember why it is not coloured. To cast types we have to use convert() one of GDS built in methods, same with string casting is made with str() from GDS not with String() now it makes sense... closing :B


Fix race condition in json decoder

I have a project spec that will occasionally fail to generate with an error like:
Parsing project spec failed: Error Domain=Unspecified error Code=0 "(null)

When I'd run xcodegen 100 times in sequence, it would typically fail 1-3 times, so I began suspecting a race condition 🕵♂. I traced it down to a block in json(atKeyPath:invalidItemBehavior:parallel:).
Concurrently populating an array is not thread-safe because the underlying buffer may change. I think what's happening is that each BlockOperation has a mutable reference to the array, making its storage non-uniquely referenced. So each time an operation executes, the array COWs and switches to a new buffer. If another operation is in the middle of writing to the previous buffer, that mutation is lost.
Confirmed by breaking at the BlockOperation's assignment statement and checking the uniqueness of the array's buffer:
(lldb) p itemResults._buffer._storage.isUniquelyReferencedNative()
(Bool) $R12 = false

TSan also catches the error:
TSan report
==================
WARNING: ThreadSanitizer: Swift access race (pid=60834)
  Modifying access of Swift variable at 0x7b0800005ab0 by thread T4:
    #0 closure #1 in Dictionary<>.json<A>(atKeyPath:invalidItemBehaviour:parallel:) <null>:2 (xcodegen:x86_64+0x1004210db)
    #1 partial apply for closure #1 in Dictionary<>.json<A>(atKeyPath:invalidItemBehaviour:parallel:) <null>:2 (xcodegen:x86_64+0x100421c3e)
    #2 thunk for @escaping @callee_guaranteed () -> () <null>:2 (xcodegen:x86_64+0x1004215c3)
    #3 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ <null>:2 (Foundation:x86_64+0x41ac4)
    #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Previous modifying access of Swift variable at 0x7b0800005ab0 by thread T1:
    #0 closure #1 in Dictionary<>.json<A>(atKeyPath:invalidItemBehaviour:parallel:) <null>:2 (xcodegen:x86_64+0x1004210db)
    #1 partial apply for closure #1 in Dictionary<>.json<A>(atKeyPath:invalidItemBehaviour:parallel:) <null>:2 (xcodegen:x86_64+0x100421c3e)
    #2 thunk for @escaping @callee_guaranteed () -> () <null>:2 (xcodegen:x86_64+0x1004215c3)
    #3 __NSBLOCKOPERATION_IS_CALLING_OUT_TO_A_BLOCK__ <null>:2 (Foundation:x86_64+0x41ac4)
    #4 _dispatch_client_callout <null>:2 (libdispatch.dylib:x86_64+0x2657)

  Location is heap block of size 24 at 0x7b0800005aa0 allocated by main thread:
    #0 malloc <null>:3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x523ea)
    #1 swift_slowAlloc <null>:2 (libswiftCore.dylib:x86_64+0x2f2ca8)
    #2 Project.init(basePath:jsonDictionary:) <null>:2 (xcodegen:x86_64+0x10045c489)
    #3 SpecLoader.loadProject(path:projectRoot:variables:) <null>:2 (xcodegen:x86_64+0x1004c4f02)
    #4 ProjectCommand.execute() <null>:2 (xcodegen:x86_64+0x10080e77a)
    #5 protocol witness for Command.execute() in conformance ProjectCommand <null>:2 (xcodegen:x86_64+0x10080ed95)
    #6 CLI.go(with:) <null>:2 (xcodegen:x86_64+0x10076442d)
    #7 CLI.go() <null>:2 (xcodegen:x86_64+0x100763a45)
    #8 XcodeGenCLI.execute(arguments:) <null>:2 (xcodegen:x86_64+0x100811205)
    #9 main main.swift:8 (xcodegen:x86_64+0x100004cb2)

  Thread T4 (tid=2113925, running) is a GCD worker thread

  Thread T1 (tid=2113922, running) is a GCD worker thread

SUMMARY: ThreadSanitizer: Swift access race (xcodegen:x86_64+0x1004210db) in closure #1 in Dictionary<>.json<A>(atKeyPath:invalidItemBehaviour:parallel:)+0x8bb
==================

Fixed by using withUnsafeMutableBuffer to force a c-style buffer while this code writes to it, and using DispatchQueue.concurrentPerform to make the closure non-escaping. Now, TSan doesn't complain and I cannot reproduce the error!

This is great, thanks @elliottwilliams! Could you please add a changelog entry as well.

This is great, thanks @elliottwilliams! Could you please add a changelog entry as well.

@yonaskolb done, thanks for the review!

@yonaskolb done, thanks for the review!


subctl benchmark throughput not working in Openhift 4.6 with globalnet

What happened:
subctl benchmark throughput not working in Openhift 4.6 with globalnet
[asuryana@localhost testday]$ subctl benchmark throughput asuryana-cluster-a/auth/kubeconfig asuryana-cluster-b/auth/kubeconfig --verbose
Performing throughput tests
Creating kubernetes clients
Setting new cluster ID "cluster-a", previous cluster ID was "asuryana-cluster-a"
Setting new cluster ID "cluster-b", previous cluster ID was "asuryana-cluster-b"
Creating lighthouse clients
Creating submariner clients
Creating namespace objects with basename "throughput"
Generated namespace "e2e-tests-throughput-gbr2v" in cluster "cluster-a" to execute the tests in
Creating namespace "e2e-tests-throughput-gbr2v" in cluster "cluster-b"
Performing throughput tests from Gateway pod on cluster "cluster-a" to Gateway pod on cluster "cluster-b"
Creating a Nettest Server Pod on "cluster-b"
Nettest Server Pod "nettest-server-podcmknw" was created on node "ip-10-0-49-192.us-east-2.compute.internal"
Pointing a ClusterIP service to the listener pod in cluster "cluster-b"
Deleting namespace "e2e-tests-throughput-gbr2v" on cluster "cluster-a"
Deleting namespace "e2e-tests-throughput-gbr2v" on cluster "cluster-b"
panic: Failed to getsubmariner.io/globalIp annotation for service test-svc-nettest-server-pod. Service "test-svc-nettest-server-pod" does not have annotation "submariner.io/globalIp" yet
Unexpected error:
<*errors.errorString | 0xc0003d21a0>: {
s: "timed out waiting for the condition",
}
timed out waiting for the condition
occurred
goroutine 1 [running]:
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark.StartThroughputTests.func1(0xc0001b06e0, 0x15a, 0xc0009103d0, 0x1, 0x1)
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark/throughput.go:38 +0x6b
github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc00059f1c0, 0x19ba860, 0x242f700, 0x0, 0xc000712340, 0x1, 0x1, 0xc000712350)
github.com/onsi/gomega@v1.11.0/internal/assertion/assertion.go:79 +0x216
github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc00059f1c0, 0x19ba860, 0x242f700, 0xc000712340, 0x1, 0x1, 0xc000792000)
github.com/onsi/gomega@v1.11.0/internal/assertion/assertion.go:48 +0xc7
github.com/submariner-io/shipyard/test/e2e/framework.AwaitUntil(0xc0008f6000, 0x4c, 0xc0005c5af0, 0xc0005c5ac8, 0x16, 0x17cf06d)
github.com/submariner-io/shipyard@v0.9.0-rc0/test/e2e/framework/framework.go:488 +0x1c6
github.com/submariner-io/shipyard/test/e2e/framework.(*Framework).AwaitUntilAnnotationOnService(0xc000181450, 0x1, 0x17cd855, 0x16, 0xc000584c00, 0x1b, 0xc000584c20, 0x1a, 0x0)
github.com/submariner-io/shipyard@v0.9.0-rc0/test/e2e/framework/services.go:111 +0x166
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark.runThroughputTest(0xc000181450, 0x0, 0x1, 0x1, 0x1)
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark/throughput.go:129 +0x513
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark.StartThroughputTests(0x1993000)
github.com/submariner-io/submariner-operator/pkg/subctl/benchmark/throughput.go:56 +0x225
github.com/submariner-io/submariner-operator/pkg/subctl/cmd.testThroughput(0x23d4320, 0xc000695aa0, 0x2, 0x3)
github.com/submariner-io/submariner-operator/pkg/subctl/cmd/benchmark.go:86 +0x5d
github.com/spf13/cobra.(*Command).execute(0x23d4320, 0xc000695a40, 0x3, 0x3, 0x23d4320, 0xc000695a40)
github.com/spf13/cobra@v1.1.1/command.go:854 +0x2c2
github.com/spf13/cobra.(*Command).ExecuteC(0x23d52e0, 0x445bca, 0x22dc9a0, 0xc00006e778)
github.com/spf13/cobra@v1.1.1/command.go:958 +0x375
github.com/spf13/cobra.(*Command).Execute(...)
github.com/spf13/cobra@v1.1.1/command.go:895
github.com/submariner-io/submariner-operator/pkg/subctl/cmd.Execute(...)
github.com/submariner-io/submariner-operator/pkg/subctl/cmd/root.go:56
main.main()
command-line-arguments/main.go:27 +0x32
What you expected to happen:
The throughput should return the measurements.
How to reproduce it (as minimally and precisely as possible):
Run  subctl benchmark throughput asuryana-cluster-a/auth/kubeconfig asuryana-cluster-b/auth/kubeconfig --verbose in Openshift 4.6 with globalnet enabled.
Anything else we need to know?:
Environment: Openshift 4.6

Submariner version (use subctl version): subctl 0.9.0.rc0
Kubernetes version (use kubectl version):
Cloud provider or hardware configuration:
OS (e.g: cat /etc/os-release):
Kernel (e.g. uname -a):
Install tools:
Network plugin and version (if this is a network-related bug):
Others:

same in KIND setup for Globalnet+LIbreswan


The json inspector for instance views shows contaminated data!

There's some data in the JSON inspector that isn't in the actual nexus payload. I guess the object is being mutated for UI purposes.

metafields
resolvedLinks
numFields

Fixed in 0.4.8


GPU GridLayer PART-3: Add NewGridLayer.

For #2084

Background

Adding NewGridLayer, a composite layer that does some prop conversion and either renders GridLayer (cpu aggregation) or GPUGridLayer.

After all testing, GridLayer becomes CPUGridLayer and NewGridLayer becomes GridLayer.

GPUGridCellLayer

Fix lighting.
Match elevation calculations with ColumnLayer.

Tested NewGridLayer's CPU and GPU paths with existing GridLayer golden image and in layer-browser example.
Testing and docs will be added in future PRs , tracked in #2084

Change List

Add NewGridLayer, fixes to gpu-grid-cell-layer.

Add a basic unit test like this

Coverage increased (+0.04%) to 62.047% when pulling 6082182d82c907ff855a0c2d46c0210bdd78d77e on GPUGridLayer3 into 0f250341e95e635157c8f1c1c6b131ff0662a85a on master.


Add JobServerOption for --jar_cache_dir

Please add a meaningful description for your change here
Pass custom cache dir to prevent
PermissionError: [Errno 13] Permission denied: '/home/.apache_beam'

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

waiting on author

Hi @s21lee  - feel free to ping this thread when your change is ready for review.

I'll put the ball in your court for now since some test suites are failing , let us know if you need help.

waiting on author

@tvalentyn Hi, thank you for your help.
I fixed errors. But the error is left. Do you have an idea fro the error?
https://github.com/apache/beam/actions/runs/11512354850/job/32047261745?pr=32033

I re-ran the tests, let's see if it still happens.

LGTM


[AzureMonitorDistro] modify sdk version to indicate if distro is used

API change check
API changes are not detected in this pull request.


fix create job occasionally unsuccessfully  from ops

What type of PR is this?

/kind bug
What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:

/lgtm

/lgtm

issue: https://github.com/kubean-io/kubean/issues/690


Update latest Ubuntu Server version in docs

Updates reference to latest Ubuntu Server LTS version from 18.04 to 20.04.

What this PR does / why we need it:
Updates the listed Ubuntu Server version in the docs to 20.04 in order to avoid confusion
Which issue (if any) this PR fixes:
Fixes #
Any other useful info:

Covered in recent doc update commits 👍

Covered in recent doc update commits 👍


what is wrong

what is wrong
version 3.3.3 shows this

version 3.3.4 shows this

3.3.4 has an issue for our project too. The change to display: table-cell and vertical-align: middle on the handles and label meant that even though the width was being calculated correctly, it would not apply in the browser because table cells can shrink.
Our workaround was to add this to our CSS:
.bootstrap-switch .bootstrap-switch-handle-on,
.bootstrap-switch .bootstrap-switch-handle-off,
.bootstrap-switch .bootstrap-switch-label {
	display: inline-block;
	vertical-align: baseline;
}

@jameshoward Thank you. CSS fix worked for me.


WIP but ready

I realize this is work in progress, but its also the most mature pure dom lib out there. Would you consider publishing this to bower?

whilst theres still alot to be desired, i've tagged and registered with bower

also managed to get a CI up and running for PR's and commits http://hades.avoidinspace.com:8080/github.com/aktowns/purescript-simple-dom/

I agree with @joneshf that purescript-dom is a misleading name.  You can't actually do anything to the dom with it.  I think modules that only expose foreign types should have "types" or "foreign types" in the name, e.g. purescript-dom-types.  This is similar to a fairly widely used convention in which a .Types module has only types.  It's so common, in fact, that when I don't see it, I expect the module to provide some kind of functionality.  This happened to me with purescript-dom.  So I disagree with @paf31 that the difference is merely cosmetic; what things are named does matter.
(Although, as an aside, @joneshf, I found purescript-events to have a misleading name as well since it has nothing to do with DOM events as far as I can tell :) )
A lot of JS UI frameworks are really useful, and to integrate them into PS using PS bindings, we need to have access to things like events (and more generally, effectful callbacks).  I've found that everything with PS is really easy and natural for the most part, but integrating with JS GUI libraries is hard because everyone is doing these things in a different way.
I don't have a solution, though.  Just a dude with an opinion and a keyboard...


Prevent a cell from being modified

Description
I want to revent a sheet from being modified.
Steps to reproduce the issue:

open protect worksheet
select Set column format Set row format Select locked cells Select unlocked cells
Want to can modify Modify cell width，but can not modify cell value
I fork the this lib. the add API GetProtectSheet, read the FormatSheetProtection, then set the save field which is true. but it not works.

Describe the results you received:
protect all, I can not modify modify cell width
Describe the results you expected:
can modify Modify cell width，but can not modify cell value
Output of go version:
go version go1.12.6 darwin/amd64

Excelize version or commit ID:
9279c86d85ab0077f3696b8ec4cfb49ad8222530

Environment details (OS, Microsoft Excel™ version, physical, etc.):
mac:
Darwin localhost 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64
excel:
v16.16.12(190715)
func (f *File) GetProtect(sheet string) (*FormatSheetProtection, error) {
	xlsx, err := f.workSheetReader(sheet)
	if err != nil {
		return nil, err
	}
	var ret FormatSheetProtection
	ret.AutoFilter = xlsx.SheetProtection.AutoFilter
	ret.DeleteColumns = xlsx.SheetProtection.DeleteColumns
	ret.DeleteRows = xlsx.SheetProtection.DeleteRows
	ret.FormatCells = xlsx.SheetProtection.FormatCells
	ret.FormatColumns = xlsx.SheetProtection.FormatColumns
	ret.FormatRows = xlsx.SheetProtection.FormatRows
	ret.InsertColumns = xlsx.SheetProtection.InsertColumns
	ret.InsertHyperlinks = xlsx.SheetProtection.InsertHyperlinks
	ret.InsertRows = xlsx.SheetProtection.InsertRows
	ret.EditObjects = xlsx.SheetProtection.Objects
	ret.PivotTables = xlsx.SheetProtection.PivotTables
	ret.EditScenarios = xlsx.SheetProtection.Scenarios
	ret.SelectLockedCells = xlsx.SheetProtection.SelectLockedCells
	ret.SelectUnlockedCells = xlsx.SheetProtection.SelectUnlockedCells
	ret.Sort = xlsx.SheetProtection.Sort
	return &ret, err
}

Hi @exfly, this issue has been fixed. Please try to upgrade the library with the master branch code.


Bitwise operators don't take advantage of JavaScript truthiness

I've been using bitmasks to consolidate multiple fields/checks and since Dart requires that all boolean conditions be typed as such, I'm required to add a != 0 check when testing for the presence of a bit:
if (_data.changeDetectorState & ChangeDetectorStates.detachedBit != 0) {
  return; // Do nothing if detached.
}

I'd imagine dart2js would take advantage of JavaScript's truthiness, and essentially compile away the != 0 check. However, I've noticed that it retains !== 0:
if ((t1._component$_changeDetectorState & 2) !== 0)
  return;

This seems like a reasonable code-size optimization.
It is unclear if it is also a performance optimization - JavaScript has many falsy values (null, undefined, 0, -0.0, 0n, "", false, ...) so the test is potentially more costly, but we would save the cost of !== 0. In the <5% of code that actually gets JIT-optimized by JavaScript, it would probably generate similar code for both.
When I looked at this a few years ago I decided that I should put my effort elsewhere since the optimization would happen less than a few hundred times in some huge applications, saving < 1k of download size. We would save more space by compiling while(true) to for(;;), which we also don't do at the time of writing.
If there is an app where this makes more that a tiny improvement it would become more worthwhile.
It would not be hard to recognize this kind of condition in SsaInstructionSelection.


Add type parameter to uriToId method and implement for albums

Thanks for working on this!
I can finish up the rest in the coming days before tagging a release.


Style the data browser

Currently is rather simplistic. If we get a logo, should appear there.

Logo now in place.


Virtual Function Unhealthy when pci address map to two interfaces

Environment
Hardware: Mellanox Technologies MT27500/MT27520 Family [ConnectX-3/ConnectX-3 Pro Virtual Function]
Branch: master
Problem
We have the virtual function enabled on SR-IOV supported device, but this network interface card has 2 ports: enp3s0 and enp3s0d0. If we don't have these two interfaces enabled SR-IOV both, it will fail when healthy check.
Here is one interface of the card with SR-IOV enabled, it has 4 virtual functions.
6: enp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:02:c9:1e:b4:60 brd ff:ff:ff:ff:ff:ff
    vf 0 MAC 00:52:44:11:22:33, vlan 4095, spoof checking off, link-state enable
    vf 1 MAC 00:52:44:11:22:34, vlan 4095, spoof checking off, link-state enable
    vf 2 MAC 00:52:44:11:22:35, vlan 4095, spoof checking off, link-state enable
    vf 3 MAC 00:52:44:11:22:36, vlan 4095, spoof checking off, link-state enable

But the interface's PCI address is mapping to 2 devices: enp3s0 and enp3s0d1.
ubuntu@master:~$ ls -al /sys/bus/pci/devices/0000\:03\:00.0/net
total 0
drwxr-xr-x  4 root root 0 Jan  8 00:21 .
drwxr-xr-x 10 root root 0 Jan  8 00:21 ..
drwxr-xr-x 10 root root 0 Jan  8 00:21 enp3s0
drwxr-xr-x 10 root root 0 Jan  8 00:21 enp3s0d1

And in the source code pkg/resources/netDevicePool.go, it will check by pf parameter, I believe it is PCI address we feed in /etc/pcidp/config.json.
for _, pf := range rc.RootDevices {
    // If the PF link is not up = "Unhealthy"
    if !utils.IsNetlinkStatusUp(pf) {
        healthValue = pluginapi.Unhealthy
    }
    ... snip ...

Then it will check pf by IsNetlinkStatusUp function, in utils/utils.go, and because we can't make sure what is the interface name for this PCI address, so we use the wildcard to find interface's operstate.
if opsFiles, err := filepath.Glob(filepath.Join(sysBusPci, dev, "net", "*", "operstate")); err == nil {
    for _, f := range opsFiles {
        bytes, err := ioutil.ReadFile(f)
        if err != nil || strings.TrimSpace(string(bytes)) != "up" {
            return false
        }
    }
}

But once we don't have both enp3s0 and enp3s0d0 keep in up status, it failed on the health check.
Further question
this is two isolated interfaces share with same pci address. When we locate VFs into the container, how can I know which interface's VF is located to the container?

@moshe010 do you think we can close this issue?

yes, the The "pfNames" selector which can specify a range of VFs should solve it.


fix: also check overflowY: overlay in detectScroll

Ref: https://developer.mozilla.org/en-US/docs/Web/CSS/overflow

Behaves the same as auto, but with the scrollbars drawn on top of content instead of taking up space. Only supported in WebKit-based (e.g., Safari) and Blink-based (e.g., Chrome or Opera) browsers.

I would have sworn it was listed already, but apparently not. Thanks for the fix!

@dwelle could you release new version, that includes this patch？

@tokuhirom sorry for replying late. It'll take some time before we release a new stable version, but you can use https://www.npmjs.com/package/@excalidraw/excalidraw-next if you need this asap


Need way to specify skip-level parent relationships

Say you have the following Resource hierarchy:

Organization

Group

User

Modeled as:
const Organization = new Tyr.Collection({
  name: 'organization',
  ...
};

const Group = new Tyr.Collection({
  name: 'group',
  fields: {
    organizationId: {
      link: 'organization',
      relate: 'ownedBy',
      direction: 'outgoing',
      graclType: [ 'resource' ]
    },
    ...
  },
  ...
};

const User = new Tyr.Collection({
  name: 'user',
  fields: {
    organizationId: { link: 'organization' },
    groupIds:       { is: 'array',
                      of: {
                        link: 'group',
                        relate: 'ownedBy',
                        direction: 'outgoing',
                        graclType: [ 'resource' ]
                      }
                    },
    ...
  },
  ...
}

If I grant a permission on the top level parent (Organization in this case), Users in that Organization but not in an intermediate Group won't inherit the permission.

I think this can be added without too much of a change to the codebase. Basically, if a collection has a parent, and no ids for the parent, I will check if the next level parent has a "path" to the original collection, if not, recurse to the next parent.

The above fix will not be totally complete however, for something like the following:
Say we have the following schema
user -> teams -> org
       `-> regions -> org
       `-> org

That is that user has teams and regions, which both have orgs, as well as a direct orgId
Additionally, lets say the gracl hierarchy is as follows
user -> teams -> orgs

finally, assume that the user doesn't have any team ids or organization id, but it does have regions.
The above fix will first look for teams, will find none, and then look for the shortest path
between the next link in the gracl hierarchy (orgs) and user, which should just be user.organizationId.
however, since there is no direct organizationId set on the user, it will find nothing, and thus return no parents and stop the recursive check.
However, we ideally would also want to use the path from user-> region -> organization. The problem
is that this essentially requires enumerating all possible paths between user and organization which
becomes an O(2^n) problem. Right now we are just storing the shortest paths, which can
be pre-computed in O(n^3) time.

Alternatively, I suppose we could use some sort of constrained pathfinding algorithm which would exclude the links that are not available (and finds an alternative path to the end collection) -- this is beyond the scope of the above specific problem though, and can be addressed later

Discussed solution to this bug implemented here: https://github.com/CrossLead/tyranid-gracl/commit/fbce79ac3e73993dff342111060e41f9be78d811


plg: display errors when autocomplete rate limits trigger

Closes #2134
This bug has 2 parts to it:

Using the rate limited demo accounts, by design the headers retry-after return back the same time as when the request was made. This causes the error messages to constantly be displayed in the quick pick view
Also using the rate limited demo account, if you hard code the retry time to a later time (like 1 day after), the error message does not display anymore after being dismissed once.

This PR adds a check to see whether or not the errors have been added in the statusBar and to display the rate limited error if no other RateLimitError is present. This also makes sure the error is only displayed once, instead of spamming the user if autocomplete keeps getting triggered.
Before (bug 1):
https://github.com/sourcegraph/cody/assets/51868853/de5a963a-a5be-4677-a75c-4f154960a516
Before (bug2, after hardcoding a later retry-after date):
https://github.com/sourcegraph/cody/assets/51868853/ffcd7748-0827-4fca-b71f-e141112ead97
After:
https://github.com/sourcegraph/cody/assets/51868853/48078429-59dc-4ad8-a744-efb1e665a689
Test plan

login to https://sourcegraph.com using the free demo account (thread)
go to any file, trigger autocompletes multiple times and you will see only 1 error message displayed
click on the error, dismiss it, and trigger another autocomplete. you should see another error message displayed

I think something with the rate limited test account might be incorrect as it says "limit will reset in less than a minute". This isn't how our rate limiter works AFAIK?

There is clearly something I misunderstand about this header 😆  Should we remove the text that depends on the retry-after value?

Getting some clarity here: https://sourcegraph.slack.com/archives/C05AGQYD528/p1701859689434749

@kalanchan pushed some tweaks (https://github.com/sourcegraph/cody/pull/2135/commits/363a716c402be4f04a49eaabb175bc7a0d2bdf5c), think this is good to merge?

@philipp-spiess @DanTup THANKS ALL for reviewing and helping, feels good to be contributing back to the team 💪


checks: add checks for required annotations

This change adds a new check for a list of required annotations. For
now, only "charts.openshift.io/name" is checked to mitigate HELM-226
(see link below).
In terms of profile, the annotation check is enforced only in partner
and redhat profile for now.
Ref: https://issues.redhat.com/browse/HELM-226
Signed-off-by: Allen Bai<EMAIL_ADDRESS>

By default the code has set the profile.vendorType=default, and profile.version=v1.0 (which is the latest default profile version). And we can specify different profile and version with the --set flag:
$ ./out/chart-verifier verify pkg/chartverifier/checks/chart-0.1.0-v3.missing-annotations.tgz --set profile.vendorType=partner --set profile.version=v1.0

apiversion: v1
kind: verify-report
metadata:
    tool:
        verifier-version: 1.2.3
        profile:
            VendorType: partner
            version: v1.0
        chart-uri: pkg/chartverifier/checks/chart-0.1.0-v3.missing-annotations.tgz
        digest: sha256:54e8a89e38bfdc20c2a365d04bf5df8712920ba85dfd9229405baf3631e031ff
        digests:
            chart: sha256:54e8a89e38bfdc20c2a365d04bf5df8712920ba85dfd9229405baf3631e031ff
            package: c2d5ee107f7860b90258d853341b16d0a521f313d5b14039edcba35dc1ab368f
        lastCertifiedTimestamp: "2021-10-18T14:25:41.323779-04:00"
        certifiedOpenShiftVersions: N/A
    chart:
        name: psql-service
        home: ""
        sources: []
        version: 0.1.7
        description: A Helm chart for a RedHat Certified PSQL
        keywords: []
        maintainers: []
        icon: ""
        apiversion: v2
        condition: ""
        tags: ""
        appversion: 10.0.0
        deprecated: false
        annotations:
            charts.openshift.io/archs: x86_64
            charts.openshift.io/provider: RedHat
            charts.openshift.io/supportURL: https://github.com/dperaza4dustbit/helm-chart
        kubeversion: 1.20.0
        dependencies: []
        type: application
    chart-overrides: ""
results:
  - check: v1.0/has-readme
    type: Mandatory
    outcome: PASS
    reason: Chart has a README
  - check: v1.0/contains-test
    type: Mandatory
    outcome: PASS
    reason: Chart test files exist
  - check: v1.0/contains-values-schema
    type: Mandatory
    outcome: PASS
    reason: Values schema file exist
  - check: v1.0/has-kubeversion
    type: Mandatory
    outcome: PASS
    reason: Kubernetes version specified
  - check: v1.0/not-contains-crds
    type: Mandatory
    outcome: PASS
    reason: Chart does not contain CRDs
  - check: v1.0/helm-lint
    type: Mandatory
    outcome: FAIL
    reason: |+
        Helm lint has failed:  [INFO] Chart.yaml: icon is recommended
        [ERROR] values.yaml: - k8Project: Does not match pattern '^[a-z0-9-_]+$'

        [ERROR] templates/: values don't meet the specifications of the schema(s) in the following chart(s):
        psql-service:
        - k8Project: Does not match pattern '^[a-z0-9-_]+$'

  - check: v1.0/chart-testing
    type: Mandatory
    outcome: FAIL
    reason: |-
        Error running process: executing helm with args "install psql-service-dudgvwh1hj /home/abai/.cache/chart-verifier/pkg_chartverifier_checks_chart_0_1_0_v3_missing_annotations_tgz/psql-service --namespace default --wait --values /tmp/chart-testing-996766246/values.yaml": exit status 1
        ---
        Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "https://api.dev-svc-4.9-101807.devcluster.openshift.com:6443/version?timeout=32s": dial tcp: lookup api.dev-svc-4.9-101807.devcluster.openshift.com: no such host
  - check: v1.0/is-helm-v3
    type: Mandatory
    outcome: PASS
    reason: API version is V2, used in Helm 3
  - check: v1.0/contains-values
    type: Mandatory
    outcome: PASS
    reason: Values file exist
  - check: v1.0/not-contain-csi-objects
    type: Mandatory
    outcome: PASS
    reason: CSI objects do not exist
  - check: v1.0/images-are-certified
    type: Mandatory
    outcome: FAIL
    reason: |
        Failed to certify images : Failed to get images, error running helm template : values don't meet the specifications of the schema(s) in the following chart(s):
        psql-service:
        - k8Project: Does not match pattern '^[a-z0-9-_]+$'

and v1.1:
$ ./out/chart-verifier verify pkg/chartverifier/checks/chart-0.1.0-v3.missing-annotations.tgz --set profile.vendorType=partner --set profile.version=v1.1

apiversion: v1
kind: verify-report
metadata:
    tool:
        verifier-version: 1.2.3
        profile:
            VendorType: partner
            version: v1.1
        chart-uri: pkg/chartverifier/checks/chart-0.1.0-v3.missing-annotations.tgz
        digest: sha256:54e8a89e38bfdc20c2a365d04bf5df8712920ba85dfd9229405baf3631e031ff
        digests:
            chart: sha256:54e8a89e38bfdc20c2a365d04bf5df8712920ba85dfd9229405baf3631e031ff
            package: c2d5ee107f7860b90258d853341b16d0a521f313d5b14039edcba35dc1ab368f
        lastCertifiedTimestamp: "2021-10-18T14:25:48.277673-04:00"
        certifiedOpenShiftVersions: N/A
    chart:
        name: psql-service
        home: ""
        sources: []
        version: 0.1.7
        description: A Helm chart for a RedHat Certified PSQL
        keywords: []
        maintainers: []
        icon: ""
        apiversion: v2
        condition: ""
        tags: ""
        appversion: 10.0.0
        deprecated: false
        annotations:
            charts.openshift.io/archs: x86_64
            charts.openshift.io/provider: RedHat
            charts.openshift.io/supportURL: https://github.com/dperaza4dustbit/helm-chart
        kubeversion: 1.20.0
        dependencies: []
        type: application
    chart-overrides: ""
results:
  - check: v1.0/contains-test
    type: Mandatory
    outcome: PASS
    reason: Chart test files exist
  - check: v1.0/helm-lint
    type: Mandatory
    outcome: FAIL
    reason: |+
        Helm lint has failed:  [INFO] Chart.yaml: icon is recommended
        [ERROR] values.yaml: - k8Project: Does not match pattern '^[a-z0-9-_]+$'

        [ERROR] templates/: values don't meet the specifications of the schema(s) in the following chart(s):
        psql-service:
        - k8Project: Does not match pattern '^[a-z0-9-_]+$'

  - check: v1.0/required-annotations-present
    type: Mandatory
    outcome: FAIL
    reason: 'Missing required annotations: [charts.openshift.io/name]'
  - check: v1.0/is-helm-v3
    type: Mandatory
    outcome: PASS
    reason: API version is V2, used in Helm 3
  - check: v1.0/contains-values
    type: Mandatory
    outcome: PASS
    reason: Values file exist
  - check: v1.0/contains-values-schema
    type: Mandatory
    outcome: PASS
    reason: Values schema file exist
  - check: v1.0/has-kubeversion
    type: Mandatory
    outcome: PASS
    reason: Kubernetes version specified
  - check: v1.0/not-contains-crds
    type: Mandatory
    outcome: PASS
    reason: Chart does not contain CRDs
  - check: v1.0/not-contain-csi-objects
    type: Mandatory
    outcome: PASS
    reason: CSI objects do not exist
  - check: v1.0/images-are-certified
    type: Mandatory
    outcome: FAIL
    reason: |
        Failed to certify images : Failed to get images, error running helm template : values don't meet the specifications of the schema(s) in the following chart(s):
        psql-service:
        - k8Project: Does not match pattern '^[a-z0-9-_]+$'
  - check: v1.0/chart-testing
    type: Mandatory
    outcome: FAIL
    reason: |-
        Error running process: executing helm with args "install psql-service-r58nd08tmz /home/abai/.cache/chart-verifier/pkg_chartverifier_checks_chart_0_1_0_v3_missing_annotations_tgz/psql-service --namespace default --wait --values /tmp/chart-testing-579638365/values.yaml": exit status 1
        ---
        Error: INSTALLATION FAILED: Kubernetes cluster unreachable: Get "https://api.dev-svc-4.9-101807.devcluster.openshift.com:6443/version?timeout=32s": dial tcp: lookup api.dev-svc-4.9-101807.devcluster.openshift.com: no such host
  - check: v1.0/has-readme
    type: Mandatory
    outcome: PASS
    reason: Chart has a README

See the profile version in the metadata and checks in v1.0 vs. v1.1.

remain providerType=default, and use provider.version to set version of default provider (e.g. partner v1.0 etc).

Added, looking at --chart-set charts.openshift.io/name=niceName <chart>


Configure cloud-9 workspace as part of template

To address access control issues around IAM role creation define cloud9 instance and associated instance role in template. This should align with #226

It's better to use a Cloud9 workspace with this paragraph https://chaos-engineering.workshop.aws/en/030_basic_content/040_ssm/050_direct_automation.html
For example, how to do that without Cloud9 and only CloudShell

Copy the YAML below into a file named ssm-terminate-instances-asg-az.yaml

Ultimately it's probably better to get rid of cloud9 from the workshop entirely because the only thing it's currently required for is the deploy mechanism and adopting the EE/codebuild approach for that as well is likely a more consistent experience overall.
However, good point @gmarchand: I take it for granted that users are familiar with tools like vi to trivially create files and that's a really bad assumption. So if we manage to switch to cloudshell this needs some verbiage / explanation.


fix(challenge): Improved tests for Algorithm "Missing Letters".

Pre-Submission Checklist

[x] Your pull request targets the staging branch of freeCodeCamp.
[x] Branch starts with either fix/, feature/, or translate/ (e.g. fix/signin-issue)
[x] You have only one commit (if not, squash them into one commit).
[x] All new and existing tests pass the command npm test. Use git commit --amend to amend any fixes.

Type of Change

[x] Small bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds new functionality)
[x] Breaking change (fix or feature that would change existing functionality)
[ ] Add new translation (feature adding new translations)

Checklist:

[x] Tested changes locally.
[x] Closes currently open issue (replace XXXX with an issue no): Closes #12610

Description
Thanks to @breatheloud for reporting and triaging this! :smile: :+1:

Improve tests to cover what the instructions actually ask for.
Update solution to pass new test suite.

Edit: This will actually break solutions that previously passed. But the requirements in the instructions still remain the same so this shouldn't really matter.

LGTM! 👏

Only a pleasure! Thanks for fixing it guys, imo it's a good thing that it will break previous solutions that weren't thorough enough. I suppose it'll require coders to revisit the challenge to complete the certificate?

@breatheloud Maybe, maybe not - but it should be a pretty easy challenge for those with JS certs completed already.
Please tell us if you find more bugs, we're getting hungry over here... :smile:


Turn off PS autohinting by default

See #838.

Thanks.
@bramstein would you like me to cut a new release for you?


{CI} Notify batch CI errors to teams channel

Related command

Description

In batch ci mode, if any job failure is detected, sent the message to the teams channel.
Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

CI


Error on publishAggregateAndProofs

Describe the bug
Found this in Ropsten
370320:May-31 02:14:08.141[API]             error: Error on publishAggregateAndProofs [0] slot=3370, index=1 code=ATTESTATION_ERROR_ATTESTERS_ALREADY_KNOWN, targetEpoch=105, aggregateRoot=0x45e8defa8a743ac4bd3a38fdcef443da6734f438b0089740b3b638a20e003895

Expected behavior
due to #4019 , our published AggregateAndProof may be known to us, but we should be able to publish to the network as it could be helpful to others

Duplicate of https://github.com/ChainSafe/lodestar/issues/4089


Eufy cameras no longer work with 1.9.7 (mse: unsupported scheme)

After changing from 1.9.6 to 1.9.7 (Docker), the streams of my two Eufy cameras no longer work.
They are a T8410 and a T8210.
The stream comes via the eufy-security-ws addon in Home Assistant.
The message mse: unsupported scheme is displayed in the video stream.
When I switch back to 1.9.6, everything works again.
Do I have to make any adjustments in the config? This was previously generated automatically.

I am right now working to get my Eufy doorbell T8210 cam that is working good in HA card view (over webrtc-camera) also forwarded to HomeKit.
during I did search here how to configure it in go2rtc for HomeKit I did notice the update to 1.9.7 and did update.
Now my eufy cam stops working also in HA itself. Not 100% sure if the root case is the update but it looks like.
Before the cam live view was working in HA card view over webrtc-camera always, and my main problem was a very slow streaming video in Apple Home app by forwarding the cam over the HomeKit integration of HA.
Now I understand that i should do it direct in go2rtc if I see it right?
But first I need to sort out if the update break my setup or not ???

Update: still not working any more at all.
this is my config that was generated during installation with the eufy addons and webrtc-camera that worked good before the update.
streams:
T8210P812419166F:
- T8210P812419166F
And I see now this warning in the log:
23:30:05,783	trace	[streams] dial cons=0 prod=0 error=streams: unsupported scheme: T8210P812419166F
23:30:05,782	trace	[streams] check cons=0 media=video, sendonly, ANY
23:30:01,904	warn	[rtsp] error=streams: streams: unsupported scheme: T8210P812419166F stream=T8210P812419166F

Well. The T8210P812419166F scheme is definitely not supported. I'm not familiar with the eufy addon and don't know how it works. And I don't know why it worked before.

I think the scheme is made up of model number and serial number.
The setup consists of the eufy-security-ws addon, the Eufy Security Integration, the WebRTC Camera Integration and Go2RTC (in some variant). The use of Go2RTC is actually included in the Eufy Security Integration documentation.
Stream provider is P2P and the stream URL is an RTSP.

I have the same problem.

Same problem here

I see the same problem. My Eufy doorbell stream no longer works in Home Assistant when using 1.9.7
Rolling back to 1.9.6 makes it work again.

And what you can see in config for old version?

And what you can see in config for old version?

That is the config as seen once I rolled back to 1.9.6

And what you can see in config for old version?

The config is the same for both versions.

If someone is a developer - it might be helpful to know which commit broke. Just need to build a binary for each commit between 1.9.6 and 1.9.7.

You mean just try out nine different builds?

Yes. Just git checkout, go run. Multiple times

OK. Looks like we know the problem #1458
All incoming sources broken.

Sounds great that you found the source of the problem.
Can someone please give me some how level instructions how to downgrade to 1.9.6 again until the problem is solved?
I am using a Pi4 with standard HAOS Image and did install all over the HA add-on store/hasc as explained in the eufy cam how's to's for HA.
I think go2rtc was installed from webrtc-camera when I installed it.
Could it be a try to uninstall webrtc-camera and reinstall it in the hope it will install again the older version from go2rtc?

Fixed in latest master version

Fixed in latest master version

Tried it out, it works.
Many thanks for the quick fix.

What do I do to fix it?
I tried to uninstall and install, restart add on and home assistant, nothing

For AlexxIT.
Seen the same issue for a bunch of cameras. Good that you potentially found the source of this, however I would like to add a portion of the log here, in case it helps:
END of File warning
12:31:53.503 WRN github.com/AlexxIT/go2rtc/internal/streams/producer.go:170 > error=EOF url=http://<IP_ADDRESS>:8123/api/camera_proxy_stream/camera.front_door?token=9eaaXXXXeead096c349d5e38ba3984a1a23e2aecc9b3d58be1902824e03b2
12:31:53.539 WRN github.com/AlexxIT/go2rtc/internal/streams/producer.go:170 > error=EOF url=http://192.168.XX.XXX:8123/api/camera_proxy_stream/camera.ingarage?token=a3e0XXXX8e20e2fb5d77eb7c8fd5bd8a82e8d52d404aa66bcc66c7346c3c9cd9
New Stream error Warning with unsupported scheme
12:32:31.493 INF [streams] create new stream url=rtsp://<IP_ADDRESS>:8554/T8423TXXXXXXXXXX
As above unsupported scheme error
12:35:21.542 WRN [rtsp] error="streams: streams: unsupported scheme: T8170TXXXXXXXXXX" stream=T8170TXXXXXXXXXX
12:39:16.282 WRN [rtsp] error="streams: streams: unsupported scheme: T8423TXXXXXXXXXX" stream=T8423TXXXXXXXXXX
I have a whole bunch of Eufy cameras and as stated none are displaying any more.
Eufy Floodlight Cam S330 (Model: T8423)
Eufy Solocam S340: (Model: T8170)
Eufy Floodlight Cam E340: (Model:  T8425)
Eufy Garage-Control Cam: (Model: T8452)
Should you need any tests done, please let me know.

thanks

Fixed in latest master version
thanks

Unsupported scheme is correct error. Eufy integration creates an invalid config. This issue should be added to the eufy project.

What should a valid configuration/schema look like? Use of entity ID?

This is incoming stream. It should have empty source.
streams:
  T8210P812419166F:

Hallo zusammen, ich bin erst seit ein paar Tagen im Thema HA unterwegs und hatte vorher bereits Eufy-Kameras. Diese wollte ich nun einbinden.
Ich hatte vorher auch diesen Fehler.

Habe dann die Masterversion heruntergeladen und installiert, leider besteht der Fehler weiterhin.
Ich habe eine Eufy S2300 (3C) Kamera.

Für Hilfe wäre ich sehr Dankbar!

Yes, that's right that it still says 1.9.7 and here please English, there are not only Germans on GitHub 😉

It depends on your camera whether you have to start its stream or whether it provides a permanent stream.
Furthermore, with the current master version, the process is exactly the same as before with 1.9.6.

hi,
the current implementation of API call is like this, it is not passing any source information, but just name, go2rtc is automatically adding the source
Input
parameters = {"name": str(self.camera.serial_no)}

Output
streams:
        T84A1P10231228E1:
          - T84A1P10231228E1

After I tried to add empty source to API call like this, the output is still same. @AlexxIT , any suggestion on adding empty source using API calls?
parameters = {"name": str(self.camera.serial_no), "src": ""}

@fuatakgun you right. Creating stream with empty sources via API not supported right now. I'll think about improving that.

Any suggestion to make it possible other than manually writing to streams?

You can continue to use the current API. There is nothing wrong there, except for the error in the log about unsupported source.

@AlexxIT ,when 1.9.8 version is released, eufy security integration can continue using this non-sense source parameter just to set the incoming stream and I can continue publishing to it, is that right?
parameters = {
            "name": str(self.camera.serial_no),
            "src": str(self.camera.serial_no)
        }

In fact, it's just a useless source for the stream. go2rtc will skip it every time it sees it. The only problem is the logging of the error message.
I haven't figured out how to redesign the API yet.

Hey @AlexxIT @fuatakgun,
this seems to be the root of all issues happening in my case at:
https://github.com/AlexxIT/go2rtc/issues/874#issuecomment-2500390431
Cheers
Jan

i have an eufy s100 wall light camera and the master with this config incl. source is not working.

Same here. No eufy (+ Instar-cam) is working since weeks.
I've read a lot and tried every tip (including the master version, of course), but unfortunately nothing works.
Any idea?

I receive hundreds of these error messages per second for my Eufy cameras with the latest master version in Home Assistant. In the next iteration/beta, is a fix in the works for fixing this?
WRN github.com/AlexxIT/go2rtc/internal/streams/producer.go:170 > error=EOF
Separately, I cannot get a consistent loading of any of the streams, between the doorbell and the solar cams. They are frequently met with: "unsupported scheme: T8214XXXXXXXX". Any suggestions for any of this would be appreciated. I can share additional details as needed.

Yes, tested with the master version and all is working fine. Thanks

how did you do that? the master (HA Addon) shows exact the same error here

HA Addon is not master branch.

the version go2rtc master is not the master branch? OK I'm fully confused now ;-)

Ah, I wasn't aware that there are several versions of the HA addon.

Anyway, it doesn't work with the master branch, factored as the add-on... I think there's some wip in the background.

What happens when you click on Error in the top right-hand corner of the stream?

@z-master42 - sadly nothing as this overlay doesn't seem to be clickable ;) Anyway thx!
https://github.com/AlexxIT/go2rtc/issues/874#issuecomment-2500390431

Yes, tested with the master version and all is working fine. Thanks

@fuatakgun
could you please tell us how? I'm at a loss ...

hello dear community :)
my local home assistant setup;

go2rtc master add-on - keep in mind to have master version. if you have already master one, you have to delete it first as it is not being updated automatically. delete and install it back to be sure that you have the latest latest master :)
eufy security integration, pointing to local go2rtc (for most of us, default values would work)
webrtc custom card from https://github.com/AlexxIT/WebRTC
my s100 camera is streaming well in this setup over P2P (there are many warnings etc but it is working)

I am not claiming that this is the perfect setup, it could be improved, but until then, this is what we have
enjoy

Thx mate! I'll give it a try.

@fuatakgun thanks for sharing. If there is going to be a go2rtc version 1.9.8 that should also work right?
In that case, I'll stick with 1.9.6 for the time being.

@fuatakgun thanks for sharing. If there is going to be a go2rtc version 1.9.8 that should also work right?
In that case, I'll stick with 1.9.6 for the time being.

Yes but it is up to @AlexxIT to release a new version. I am sure all fixes will be included in next release
@AlexxIT when do you expect to release new version?

Same problem for me. Worked by downgrading version to 1.9.4 from 1.9.7.
thank you.

Same problem for me. Worked by downgrading version to 1.9.4 from 1.9.7. thank you.

How did you downgrade within HA?

Same problem for me. Worked by downgrading version to 1.9.4 from 1.9.7. thank you.

How did you downgrade within HA?

I am using docker container. Removed old container, changed version number and done.

Same problem for me. Worked by downgrading version to 1.9.4 from 1.9.7. thank you.

How did you downgrade within HA?

if you have the old version of go2rtc and if you have backed it up, you can downgrade using Home Assistant. Or else, there is no easy path

I have a list of 18 of my projects that I devote my free time to one by one. There are no definite plans for dates unless there are critical issues.

@ fuatakgun:   i've tried your steps, but it´s not working for me.
@ AlexxIT: i thank you for your time and skills, but -with all due respect and understanding-  not working security cams (in ha) are critical!  i hope you don't get me wrong.

If you have that critical security cams, I would recommend a professional survilance solution and nothing self build

As I understand it, the problem is solved with two clicks. Stop release go2rtc addon and start master go2rtc addon.
I don't know why the HA GUI doesn't provide for a quick rollback of the addon to any preferred version.
A very obvious feature that HACS has.

As I understand it, the problem is solved with two clicks. Stop release go2rtc addon and start master go2rtc addon.

Hi AlexxIT, thx for your reply!
I've tried this many times, but unfortunately it doesn't work.

So you have some other problem and the release won't change anything. Because it is not different from the master version.

Hi @AlexxIT and @fuatakgun, thanks for your replies.
I generally do not have issues with my doorbell, thanks for that. In terms of receiving hundreds of warnings per second, I just want to confirm that this model is supported along with the firmware version and that they are acceptable:
T8160
Firmware: <IP_ADDRESS>
Thanks for all that both of you do!

Hey guys,
this minimal yaml config is working (on a dashboard) with the Add-On master release.
type: custom:webrtc-camera
entity: camera.buero1
poster: image.buero1_event_image
shortcuts:
  - name: Play
    icon: mdi:play
    service: eufy_security.start_p2p_livestream
    service_data:
      entity_id: camera.buero1
  - name: Stop
    icon: mdi:stop
    service: eufy_security.stop_p2p_livestream
    service_data:
      entity_id: camera.buero1
ptz:
  service: eufy_security.ptz
  data_left:
    entity_id: camera.buero_1
    direction: LEFT
  data_right:
    entity_id: camera.buero_1
    direction: RIGHT
  data_up:
    entity_id: camera.buero_1
    direction: UP
  data_down:
    entity_id: camera.buero_1
    direction: DOWN

But I can also verify that warnings are flooding the log @AlexxIT @fuatakgun @roflware

Cheers
Jan

I am using the latest master version in a Docker container in conjunction with the WebRTC Camera card and the Eufy addon and integration and I have no ongoing warnings in my log.

After changing from 1.9.6 to 1.9.7 (Docker), the streams of my two Eufy cameras no longer work. They are a T8410 and a T8210. The stream comes via the eufy-security-ws addon in Home Assistant.
The message mse: unsupported scheme is displayed in the video stream.
When I switch back to 1.9.6, everything works again.
Do I have to make any adjustments in the config? This was previously generated automatically.

Hi. How does that work: "After changing from 1.9.6 to 1.9.7 (Docker)"?

Hi. How does that work: "When I switch back to 1.9.6, everything works again."? And is it possible to install an older version of Home Assistant [Add-on]?

Possibly if you do a local install? Create the folder in the right directory in Home Assistant, wget the URL, unpack the zip, chmod it, and then restart HA. You should be able to find it in the add-ons section under local add-ons and then install it that way. This is just a guess since I installed a different Eufy add-on that way.

If you do not know this, you are not using the Docker version.
To get the latest version in Home Assistant, install the master version of the Home Assistant add-on. It's all here.

If you do not know this, you are not using the Docker version.
To get the latest version in Home Assistant, install the master version of the Home Assistant add-on. It's all here.

Thank you, but the lastest verion does not work.

Hi @AlexxIT and @fuatakgun, thanks for your replies.
I generally do not have issues with my doorbell, thanks for that. In terms of receiving hundreds of warnings per second, I just want to confirm that this model is supported along with the firmware version and that they are acceptable:
T8160 Firmware: <IP_ADDRESS>
Thanks for all that both of you do!

Hey @fuatakgun, can you please confirm that the above-mentioned model and firmware should be supported?
Specifically:

eufyCam S330 (T8160), firmware: <IP_ADDRESS>
eufyCam S330 (T8160), firmware: <IP_ADDRESS> (it says it's up to date, not sure why it can't match the other firmware)

Thanks!

@roflware , it is not realistic to ask me to confirm if a specific device/firmware is working. I own a couple of eufy cameras, I cannot confirm it. Only other users can confirm it.
Moreover, this specific issue is about go2rtc, not directly related to eufy.

@roflware if you have go2rtc master, latest eufy-security add-on and integration, that is all.
I am responding to 10 different users per week and I am not fine to say "look at readme" anymore, because it is supposed to be the first thing to do.
Lastly, if it is working sometimes but not always, it is integration related, not go2rtc.

@roflware if you have go2rtc master, latest eufy-security add-on and integration, that is all.
I am responding to 10 different users per week and I am not fine to say "look at readme" anymore, because it is supposed to be the first thing to do.
Lastly, if it is working sometimes but not always, it is integration related, not go2rtc.

@fuatakgun: Regarding device support, it wasn't really clear and took some digging, so there could be some improvements there.
The only thing I can glean from this thread is that given the inconsistencies with actually working, this seems to be a go2rtc issue with the s330 cameras, given that the doorbell works relatively fine.

Hey @AlexxIT, I think I will finally be in a good place with my cameras after figuring this bit out: the Eufy cameras apparently randomly serve up either WebRTC (appears as RTC on the stream itself), MSE, or MJPEG. Any time MJPEG is selected, it takes forever, crashes, or artifacts to a green or gray screen. Via go2rtc, is there a way for me to only enable it to serve WebRTC and MSE only and disable MJPEG from being served? Controlling it from the HA card still doesn't avoid go2rtc serving MJPEG anyway, because if I specify "mode: webrtc,mse", it gives me an error. But if I could guarantee WebRTC or MSE from go2rtc, it would consistently work well for me.

By default, the mode parameter is set to webrtc,mse,hls,mjpeg. Because the card doesn't know in advance what technologies your browser and stream supports. Changing the mode parameter should help select only the preferred technologies. I don't know why it might not work for you.
Also, the stream usually comes with the H264/H265 codec - so only webrtc/mse/hls will work for you.
Or stream comes with MJPEG codec - so only mjpeg will work for you.
Support for both codec types is a very-very rare situation.

That last comment is really interesting. I know that I've seen both RTC/MSE and MJPEG, with the latter being very choppy and terrible. I'm not sure what causes that to happen.
But okay, it sounds like this can only be determined by the card, understood.

@roflware , are your cameras supporting RTSP natively or is it handled by integration?

@roflware , are your cameras supporting RTSP natively or is it handled by integration?

This is via the integration by bropat, what I mentioned before. It serving up RTSP would be a completely different conversation, but I imagine that Eufy doesn't support it since it's solar powered.

If it is coming from eufy-security addon and eufy_security integration, it will always generate rtsp, never webrtc.
@AlexxIT does not have full picture of our custom stream generation implementation.

I clearly don't know the entire flow here, but all I do know is that when referencing the the entity, it is streamed either as WebRTC, MSE, or MJPEG. This is done via eufy-security-ws, Eufy Security integration, and I guess it's handled via go2rtc thereafter and then displayed via the HA card with WebRTC after that. I see in go2rtc that there are HTTP links/streams specifying different modes, but I haven't figured out how to consume anything RSTP. It wouldn't be of value to me in that way anyway, since the cameras need a trigger in order to be enabled since they're solar powered. RTSP inherently implies constant on, which these cameras are not, due to power restrictions.

Any updates on this?


UI fixes for pagination and runtime column in dispatch list

[ ] I have added the tests to cover my changes.
[ ] I have updated the documentation and CHANGELOG accordingly.
[ ] I have read the CONTRIBUTING document.

@AlejandroEsquivel Please review and help us merge this.
cc : @mshkanth @santoshkumarradha

@Prasy12 As far as I know, the diff coverage needs to hit 95% in order for this to be merged.

@FyzHsn For the front-end we have raised one PR which will cover only 70-705% of the code. So this test might eventually fail when it runs for thef frontend code.
Even during earlier PRs this test fails but we usually merge it after review. The frontend code coverage is still WIP.
cc : @santoshkumarradha @mshkanth @AlejandroEsquivel


force param for cluster list

#1477  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Update Saga README.md and fix some bugs.

Signed-off-by: MabinGo<EMAIL_ADDRESS>

Coverage remained the same at 94.097% when pulling 9ea61f01b4e55d1ba28b2f03ff78f5d6acc795c9 on MabinGo:updateREADME into bc389c76b48ea5db0143e9973ab9b7aa4dcf7b29 on ServiceComb:master.

Coverage remained the same at 94.097% when pulling cb1a5ac3bb0bae8dcde65aa69aa20c3a9be4e35c on MabinGo:updateREADME into bc389c76b48ea5db0143e9973ab9b7aa4dcf7b29 on ServiceComb:master.

Coverage remained the same at 94.097% when pulling 2f23e64283afdf6b9faf764239a9b7c74244e3d7 on MabinGo:updateREADME into bc389c76b48ea5db0143e9973ab9b7aa4dcf7b29 on ServiceComb:master.


Add silex as a git dependency

Will work only once https://github.com/epfl-lara/silex/pull/3 is merged.


WIP: RCP for cosmoflow_ref_2048

This adds an RCP for cosmoflow at batch size 2048.
Unfortunately it's still in progress and I only have 11 runs done (our system has been experiencing hardware failures). I'll update this when I get the rest.
This is a draft.

closing inactive PRs, please open a new PR if needed. thanks


Is it safe to clear all entries when apply snapshot?

Hi, I'm implementing my storage, but there are some code I don't understand.
I'm not sure why we can discard all entries when apply snapshot.
If it's an outdated snapshot, there may have uncommitted entries after the snapshot point.
https://github.com/tikv/raft-rs/blob/a5423b241ed641ca0941c816153a9dfc0fdbfad9/src/storage.rs#L197-L214
As the raft paper said,
"If instead the follower receives a snapshot that describes a prefix of its log (due to retransmission or by mistake),
then log entries covered by the snapshot are deleted but entries following the snapshot are still valid and must be retained."
And I didn't find logic to prevent this when peer handling MsgSnapshot, did I miss something?

The abovementioned apply_snapshot function is usually used in tests to set up the MemStorage before the test logic.

If instead the follower receives a snapshot that describes a prefix of its log (due to retransmission or by mistake)

In this case, raft-rs drop the snapshot and fast forward the commit index to snapshot.index
https://github.com/tikv/raft-rs/blob/a5423b241ed641ca0941c816153a9dfc0fdbfad9/src/raft.rs#L2524-L2538

Thanks! I saw this but didn't get it.  Sorry for the trouble. = =


Update scala-collection-compat to 2.7.0

Updates org.scala-lang.modules:scala-collection-compat from 2.6.0 to 2.7.0.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (2.6.0).
You might want to review and update them manually.
docs/website/yarn.lock

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-lang.modules", artifactId = "scala-collection-compat" } ]

labels: library-update, early-semver-minor, semver-spec-minor, old-version-remains, commit-count:1

Codecov Report

Merging #1154 (ac52b76) into main (97c3eaf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1154   +/-   ##
=======================================
  Coverage   58.25%   58.25%
=======================================
  Files          74       74
  Lines        2446     2446
  Branches       81       81
=======================================
  Hits         1425     1425
  Misses       1021     1021

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 97c3eaf...ac52b76. Read the comment docs.


change CoreDNSDown to monitor coredns job

CoreDNSDown currently looks at kube-dns metric and not coredns

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[ ] Title of the PR starts with chart name (e.g. [stable/mychartname])

/assign @gianrubio

Please see the note at the top of the file you're modifying.  Also, you must sign your commits.

Signed-off-by: Ira Shumacher<EMAIL_ADDRESS>

Rules are sourced from another repository as described here: https://github.com/helm/charts/blob/master/stable/prometheus-operator/README.md#developing-prometheus-rules-and-grafana-dashboards
The intention behind kube-dns being the name of the service is that this is the kubernetes DNS service and the specific implementation is not relevant. Common Kubernetes deployment processes take this stance, so that is the reason why the above-mentioned rules repository does this as well. If your specific instance has another name it is likely because your installation process is different. If you would like to customize this option, or other rules configurations the best option you have is to deploy the rules separately to this chart.
/hold


why my coco_output.txt (output of evaluation coco object detection with tflite models)is empty!!!!!!

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):no
OS Platform and Distribution (e.g., Linux Ubuntu 16.04):ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:samsung s8 pluse
TensorFlow installed from (source or binary):source
TensorFlow version (use command below):1.15
Python version:3.6
Bazel version (if compiling from source):0.26.0
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version:NO CUDA
GPU model and memory:NO GPU

In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thanks!

@Davari393,
In order to expedite the trouble-shooting process, please provide a code snippet to reproduce the issue reported here. Thanks!

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!