Gather and restore GW nodes state

Add functions that will gather the state of the gateway nodes on all
clusters and restore it when required.
Will be used in redundancy tests.

The "GatherClustersGatewaysState" function will execute during
"BeforeSuite" execution and store the GW state globally.
The "RestoreClustersGatewaysState" will be called by redundancy tests
and will restore the state of GW nodes to the initial state.

The change will replace a static definition of GW state during restore.

@tpantelis
You're right.
But this change relevant for the local (kind) environment or any other where the MachineSet is not applied.
And it reflects the current failure of the kind e2e execution where an environment of two gateway nodes is created (the default is one single gateway node), the gateway restore tries to make a restore of a single gateway node only.
And in case the cluster contains two GW nodes, the restore will fail.
https://github.com/submariner-io/shipyard/blob/devel/test/e2e/framework/gateways.go#L189
Beside the gateway node fail-over test the we spoke about in the slack thread, the gateway label change is made in "[expansion] Test expanding/shrinking an existing cluster fleet" test, which I'm not sure I'm going to refactor.
https://github.com/submariner-io/submariner/blob/devel/test/e2e/cluster/add_remove_cluster.go#L90
What do you think?

But this change relevant for the local (kind) environment or any other where the MachineSet is not applied.
And it reflects the current failure of the kind e2e execution where an environment of two gateway nodes is created (the default is one single gateway node), the gateway restore tries to make a restore of a single gateway node only.
And in case the cluster contains two GW nodes, the restore will fail.
https://github.com/submariner-io/shipyard/blob/devel/test/e2e/framework/gateways.go#L189

I'm not sure I follow. I thought we agreed on slack that we don't want to modify GW labels to effect node fail-over. But this PR is related to that. I proposed for 0.14 that we introduce a new submariner.io/gateway-disabled label to prevent a node from becoming a GW. As part of test cleanup, we would just ensure no node has the label. This way we don't touch the submariner.io/gateway label and conflict with the ACM controller. This approach may not work when MachineSets are used.

Beside the gateway node fail-over test the we spoke about in the slack thread, the gateway label change is made in "[expansion] Test expanding/shrinking an existing cluster fleet" test, which I'm not sure I'm going to refactor.
https://github.com/submariner-io/submariner/blob/devel/test/e2e/cluster/add_remove_cluster.go#L90

Those tests were pretty much experimental and don't run (ie they're pending).

Ok, if the second test that is using the labeling is experimental and not really executed, I'll close this PR and start working on a flow we discussed on slack.


⌘-k not clearing the terminal

Versions
VS Code: Version 1.15.1 (1.15.1) | 41abd21afdf7424c89319ee7cb0445cc6f376959 | 2017-08-16T17:11:22.564Z
macOS Sierra: Version 10.12.3
Atom Keymap version: 3.0.1

Problem
⌘-k does not clear the terminal as expected
Steps to Reproduce

Install and activate "Atom Keymap" ("Code" > "Preferences" > "Keymap Extensions (⌘-K ⌘-M)")
Open integrated terminal and ensure it has focus (⌘- ![firefox-nightly-57-about-logo](https://user-images.githubusercontent.com/7552754/29482008-11f040c8-8446-11e7-9c97-ad216c3ede8f.png) )
Press ⌘-k to trigger the clear command

Expected
Clears terminal
Actual
Shows message about chord and does not execute clear:

Notes

(a) Disabling the Atom Keymap, (b) removing the "atomKeymap.promptV3Features": true, from my settings, and (c) restarting VS Code fixes the problem, but of course, doing so leaves me without the Atom Keymapping. :-(
related discussion (such that it was): https://github.com/Microsoft/vscode/issues/25059

@chrisdias @auchenberg @seanmcbreen assigning to all of you, the extension was maintained by @waderyan in the past.

This does indeed reproduce with keymap + default settings. If cmd+k is set as a user setting it works. I assume some default keybindings are being overridden as the keymap defines cmd+k + keys.

For those of you who are okay with hacks. This is how I resolved the issue.

I'll go with @am17torres HACK while this issue is not resolved

I just started trying VSCode (coming from Atom) and experienced the same issue of Terminal not clearing. Any updates on this?

fixed


Bot often reconnects to IRC and stops working when it receives ERR_NOTREGISTERED (QuakeNet)

Bot connects to Discord and IRC, relays messages, but after some time it reconnects to IRC, after some more time it disconnects from IRC (EOF from client) and in the logs it says following thing:
2019-08-12T11:01:54.187Z error: Received error event from IRC {
  prefix: 'euroserv.fr.quakenet.org',
  server: 'euroserv.fr.quakenet.org',
  command: 'err_notregistered',
  rawCommand: '451',
  commandType: 'error',
  args: [ 'dbot', 'dbot', 'Register first.' ],
  level: undefined,
  message: undefined,
  timestamp: '2019-08-12T11:01:54.187Z',
  splat: undefined,
  [Symbol(level)]: 'error',
  [Symbol(splat)]: [
    {
      prefix: 'euroserv.fr.quakenet.org',
      server: 'euroserv.fr.quakenet.org',
      command: 'err_notregistered',
      rawCommand: '451',
      commandType: 'error',
      args: [ 'dbot', 'dbot', 'Register first.' ]
    }
  ]
}

Config is as follows:
[
  {
    "nickname": "dbot",
    "server": "euroserv.fr.quakenet.org",
    "discordToken": "REDACTED",
    "autoSendCommands": [
      ["AUTH", "dbot", "REDACTED"]
    ],
    "channelMapping": {
      "#server-chat": "#jeff-xonotic"
    },
    "ircOptions": {
      "encoding": null
    }
  }
]

Could you enable debug mode, by putting "debug" : true into ircOptions and starting the bot with the environment variable NODE_ENV=development? I'm not clear on why the authentication would work some times but not others, and hopefully the logs will show what went on in the auth process to cause this. (Note that your logs will get very verbose, and you may need to redact passwords or tokens from the auth process. The auth process should contain the line AUTH dbot redacted, possibly with some colons between words.)

I enabled debug mode and I am currently waiting for it to stop working, it takes some time

log.txt

I'm not sure what's causing the underlying disconnections – the ping that the bot sends to the server isn't being responded to, so we get a timeout. (The very first line of the log shows that we do get PONG responses sometimes, but they stopped after that line and never resumed.) The reason the bot then doesn't continue to reconnect, after the ten reconnections, is that we've hit the automatic retry cap (this defaults to 10 in discord-irc). The log you just uploaded here doesn't contain err_notregistered, so that makes me think that that specific error in the original issue text was a fluke and is unlikely to repeat.
You could ensure the bot will keep reconnecting by adding "retryCount": null to your ircOptions object in the configuration (so it will retry an unlimited amount). This might help if the issue is just down to a few minutes of spotty network connection, though it will probably still disconnect and reconnect as frequently as it has been doing. Maybe try that? In the meantime, I'll see if I can reproduce the issue when I connect to quakenet.

It shouldn't be any network issues, because there is another IRC bot run on same machine, connecting to euroserv.fr.quakenet.org too and it doesn't have any issues. Anyway I ran it with retryCount null to see if it helps. If it doesn't I might as well write a patch which restarts the bot when it disconnects, because that solves the issue temporaily

Well, I just managed to get disconnected from euroserv.fr.quakenet.org both with my bot and with my actual IRC client, so I'm gonna say that it's very likely an issue with the server. Maybe your other IRC bot is just really permissive about ping timeouts? Have you tried using a different server to connect to quakenet?

Or maybe other bot is set up for other quakenet than euroserv.fr (wasn't me setting it up), I'll change it and let you know

I'll close this for now – I assume that since there hasn't been any follow-up, it was a problem with the IRC server as suspected.
If this is still happening, feel free to get back! =)


doctor issue , cannot read . system was removed form the environement variable

Is there an existing issue for this?

[X] I have searched the existing issues
[X] I have read the guide to filing a bug

Steps to reproduce
Flutter (the doctor check crashed)
    X Due to an error, the doctor check did not complete. If the error message below is not helpful, please let us know about this issue at https://github.com/flutter/flutter/issues.
    X Exception: Cannot find the executable for `where`. This can happen if the System32 folder (e.g. C:\Windows\System32 ) is removed from the PATH environment variable. Ensure that this is present and then try again after restarting the terminal and/or IDE.
    • #0      throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
      #1      _WindowsUtils._which (package:flutter_tools/src/base/os.dart:488:7)
      #2      OperatingSystemUtils.which (package:flutter_tools/src/base/os.dart:92:31)
      #3      FlutterValidator._validateSdkBinary (package:flutter_tools/src/doctor.dart:617:52)
      #4      FlutterValidator._validateRequiredBinaries (package:flutter_tools/src/doctor.dart:604:47)
      #5      FlutterValidator.validate (package:flutter_tools/src/doctor.dart:518:7)
      #6      Doctor.startValidatorTasks.<anonymous closure> (package:flutter_tools/src/doctor.dart:258:72)
      #7      asyncGuard.<anonymous closure> (package:flutter_tools/src/base/async_guard.dart:111:32)
      #8      _rootRun (dart:async/zone.dart:1399:13)
      #9      _CustomZone.run (dart:async/zone.dart:1301:19)
      #10     _runZoned (dart:async/zone.dart:1804:10)
      #11     runZonedGuarded (dart:async/zone.dart:1792:12)
      #12     runZoned (dart:async/zone.dart:1744:12)
      #13     asyncGuard (package:flutter_tools/src/base/async_guard.dart:109:3)
      #14     Doctor.startValidatorTasks (package:flutter_tools/src/doctor.dart:250:9)
      #15     Doctor.diagnose (package:flutter_tools/src/doctor.dart:378:72)
      #16     DoctorCommand.runCommand (package:flutter_tools/src/commands/doctor.dart:49:48)
      #17     FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:1616:12)
      <asynchronous suspension>
      #18     FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1297:27)
      <asynchronous suspension>
      #19     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #20     CommandRunner.runCommand (package:args/command_runner.dart:212:13)
      <asynchronous suspension>
      #21     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:339:9)
      <asynchronous suspension>
      #22     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #23     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:285:5)
      <asynchronous suspension>
      #24     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:115:9)
      <asynchronous suspension>
      #25     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
      <asynchronous suspension>
      #26     main (package:flutter_tools/executable.dart:90:3)
      <asynchronous suspension>

Actual results
flutter does not work
Logs

Logs

<!-- Paste your logs here -->

Flutter Doctor output

Doctor output

<!-- Paste your output here -->

X Exception: Cannot find the executable for where. This can happen if the System32 folder (e.g. C:\Windows\System32 ) is removed from the PATH environment variable. Ensure that this is present and then try again after restarting the terminal and/or IDE.

This looks like you have forgotten this part: Update your path. Please follow step-by-step there to see if it helps. You can also check this SO thread as well: https://stackoverflow.com/questions/49609889/flutter-doctor-doesnt-work-on-neither-command-prompt-or-powershell-window
Your question is best suited for support channels like StackOverflow, so please ask the same there for solution if any. Closing for now. If there isn't any solution that can apply to the issue, please write in comments with more information and I will re-open issue for further support. Thanks!


Getting Discord Webhook Notification but NO Web Panel Connection

exactly what the title says. ran the app. it seems to connect but can't use commands because they're not visible!

share the apk here


How to hide loading indicator in AsyncPaginatedDataTable2?

I have less data than the number of rows per page.

Do you have those circles spinning all the time? Do you have a small reproduction repo that you can share for me to look at?

I think it's when the loaded data has a length < totalCount

This happens when, rowsPerPage is lets say 4
and actual rows returned is 2
This scenario usually comes at the last page, cause we might have total 10 records. on first two pages 4+4 =8 records are shown.
but on 3rd page we only have 2 rows to show, whereas it expects us to be returning 4 rows.
Any workaround ? cc: @maxim-saplin


thinking-in-graph images

About the images, i can improve the resolution of the sections 'It is semantic' and 'It is alive' :
make higher resolution of the pics or create original ones. let me know

Sounds good (about the pics ;))


Update documentation for new Narwhal install

Summary
All in the title. This version of spack-stack is identical to what was installed on Navy's Nautilus platform. Rendered documentation is here: https://spack-stack--683.org.readthedocs.build/en/683/PreConfiguredSites.html
Testing
n/a
Applications affected
None of the existing applications is affected, installed in parallel to spack-stack-1.4.0 installation.
Systems affected
Navy's Narwhal system
Dependencies
n/a
Issue(s) addressed
n/a
Checklist

[x] This PR addresses one issue/problem/enhancement, or has a very good reason for not doing so.
[ ] These changes have been tested on the affected systems and applications: to be done by NRL colleagues in the following weeks (but tested on Nautilus)
[x] All dependency PRs/issues have been resolved and this PR can be merged.

@AlexanderRichert-NOAA @srherbener Please merge this if it gets approved while I am away (Jul 22 - Aug 13). Thanks!

@sking112 please review when you get a chance (should be quick since it's only a couple of lines of documentation). Just want to make sure someone at NRL has looked this over. Thanks!

@sking112 thanks for your quick response!


feat(amazon-eks-pod-identity-webhook): update appVersion and bum chart version

Hey @jkroepke 👋
Small PR to update the app version of the aws-eks-pod-identity-webhook helm chart and in the mean time fixing the port that is used with that new version to do the liveness/readiness healthcheck (more info here)
Don't hesitate to reach me if needed 😉
Cheers ☀

Thanks!


Fix pipeline webhook validation error

Include tekton pipelines webhook validation fix from downstream tekton results

@sayan-biswas You need to update line 6 as well.
Strangely GitHub does not allow me to comment on a line which is not changed in the PR. @gabemontero you please confirm that?

Lost some time in the CI for this.. 😣

/lgtm

/test pipeline-service-upgrade-test

same results issue on the latest downgrade test https://console-openshift-console.apps.stone-prd-rh01.pg1f.p1.openshiftapps.com/k8s/ns/tekton-ci/tekton.dev~v1beta1~PipelineRun/pipeline-service-upgrade-test-mtrr8/logs/tests-downgrade @sayan-biswas @Roming22
[results]
  - Run pipeline: OK
  - Results in database:
    - Test records: Failed
[ERROR] Unable to retrieve records for 8a01f1c3-bee1-45c6-8add-3d6d6efc0c27 from pipeline run pipelinerun.tekton.dev/test-pipelinerun-pgvj8

The prior speculation I recall from @sayan-biswas is that a timing issue with pruning is the culprit.  Certainly we have seen this across many PRs @Roming22 so I'm going to merge this, so we can get going on the staging update.
That said, it is happening enough IMO that we should open a tracker that include figuring out, we don't know already @Roming22, how to run the upgrade / downgrade locally, and debug this live, as debugging post mortem is proving too difficult.
I'll bring this up on Thursday's standup.

@gabemontero Yes, this is showing up often now, and the fact that it is mostly happening in the upgrade tests warrants an investigation. If I get the bandwidth I'll look into it in this sprint.


Update aws-java-sdk-bundle to 1.12.761

About this PR
📦 Updates com.amazonaws:aws-java-sdk-bundle from 1.12.756 to 1.12.761
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.amazonaws", artifactId = "aws-java-sdk-bundle" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.amazonaws", artifactId = "aws-java-sdk-bundle" }
}]

labels: test-library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #970.


Incomplete Instructions for Playbook Attributes

Describe the bug
During the NIST 800-53 solution installation, we are prompted to enter the TeamsChannelID and the TeamsgroupID, but there is not any information about the type of data that is expected. where do we get these values from? These attributes are not used in the Teams user interface, so I'm not sure what to enter
Screenshots
If applicable, add screenshots to help explain your problem.

Here's an article which provides a quick overview on how to obtain this data which is sourced through the "Get a link to the Team" option in the ellipses next to the team name. https://www.c-sharpcorner.com/blogs/how-to-fetch-the-teams-id-and-channel-id-for-microsoft-teams#:~:text=To fetch the teams and channel id we,be used in graph API as mentioned below

Thanks, it would be helpful to included those instructions in the workbook as a tooltip or something

Will do and thanks for your feedback!


ASP.Net Publish option is getting error

Publish getting error - solved

This solution will solve publish error.
But will not fix running error in VS first time running related to ... not found


Real-Time Inference

I noticed that in all demo scripts the video is first processed in whole and then propagated, would it be possible to run a video in real-time?
Thank you in advance!

Thanks for you issue, I think you can try to use smaller SAM 2 model for faster inference, and maybe you can refer to SAM 2's official repo to see if there're some acceleration methods on this issue

@rentainhe sorry, maybe I expressed myself badly, what I meant was if would be possible to run grounded sam2 end to end, like in a stream. Imagine I have a real time video stream and I wanted to segment it live while keeping the track of each object, would it be possible?

I think SAM 2 now only supports non-stream video input. I have no idea how to support streaming input at this time. Maybe we can refer to the community to see if there are some solutions for this issue.

@Henistein, I hope my implementation will be helpful, though these is latency.
https://github.com/patrick-tssn/Streaming-Grounded-SAM-2

Hi, @Henistein
Can your code take prompts like the grounded sam2? I mean I want to track road amd cars in a video with continuous id? Is it possible by your implementation??
TIA


OSError: [Errno 99] Address not available

I am new to home assistant so this may be my issue rather than the add on.  When I start the addon I see the following in the log ... I am unsure why I would get an OS error on the address
[s6-init] making user provided files available at /var/run/s6/etc...exited 0. [s6-init] ensuring user provided files have correct perms...exited 0. [fix-attrs.d] applying ownership & permissions fixes... [fix-attrs.d] done. [cont-init.d] executing container initialization scripts... [cont-init.d] done. [services.d] starting services [services.d] done. MQTT found in this Home Assistant instance. Starting ginlong-wifi-mqtt.py... Configuring Home Assistant... Traceback (most recent call last):   File "/ginlong-wifi-mqtt.py", line 108, in <module>     sock.bind((listen_address, listen_port)) OSError: [Errno 99] Address not available [cmd] /run.sh exited 1 [cont-finish.d] executing container finish scripts... [cont-finish.d] done. [s6-finish] waiting for services. [s6-finish] sending all processes the TERM signal.
I have the Listen address set to <IP_ADDRESS> and the port 9999 .. I have set up server b in the data stick and added the ha server ip - both are fixed IP addresses (the data stick configured as fixed, the ha server as reserved). I am able to logon to the data logger with the browser.
I am running HA in a virtual box under windows

Hi @snigehere ,
It sounds like you have setup your data stick correctly so it will broadcast data over UDP to your HA instance however try setting the listen address in the plugin to <IP_ADDRESS>

Hi - thanks for the update  .. I changed the listener to <IP_ADDRESS> and no longer get the OSError message (still no data but no error now).

The stick should send data to the registered servers every 6 minutes by default. Make sure it has been rebooted since adding your HA IP and port!

A couple more things to check:

In Advanced -> working mode I have it set to ''Data Collection'
Server B I have setup with a blank domain and connection set to TCP. There is also a test button if you want to check it is managing to send data to the plugins TCP listener.

Failing that it might be that you are using a version of the stick or firmware not supported. I only have my setup for testing but I know there are many variants which all send a different identifier code and data payload.
For reference the firmware version of my stick is: H4.01.44Y4.0.01W1.0.19(2014-11-101)

Thanks for the link - I will have a look and see if I can get it working.  So far I have just used the GUI part of home assistant and not gone any deeper so I have a steep learning curve ahead - my last coding role was 20 years ago and with lotus script!  I maybe some time before I get back.

@snigehere , no problem, have fun! :+1:

So just to update ... I found another post that suggested looking at /status.html  .. looking at the page source I can see all the data I need to get out .... Leaving off the style definitions at the top of the page this is the content ... I don't know if its of interest or not  .... so far I have not seen any data coming showing up in the mqtt broker.
One person commented that the server b does not work on the latest microcode so suggested altering the alternate one for the ginlong cloud in the config_hide.html page but not sure if that will work or will only kick in if the primary cloud fails ...  some more testing once the sun comes up again.
I guess one option would be to keep refreshing the status page and scaping the content from the page.  I would need to check if I can send the login in the page request.
`

-

+

SSID

+

`

@snigehere thanks for the update.
Scraping the status.html page is definitely one way of getting the data although plagued by the same inconsistencies between firmware as the TCP data with the added complication that the wifi stick isn't publishing the data to a TCP server lying in wait, instead we have a piece of code polling the web page including during night time hours when it won't be available leading to lots of timeouts. It will always be preferable to process the TCP packet sent from the wifi stick but that does involve some reverse engineering of the data packet to understand it.


Use strict mode in arrow function

Documentation URL
https://biomejs.dev/linter/rules/no-redundant-use-strict/
Description
I have the following IIFE
(() => {
    "use strict";
})();

and biome suggest that it's Redundant use strict directive. But I didn't see an explanation regarding this case in the documentation, and after searching on Google I haven't found much discussion on this topic. Can anyone teach me why this is redundant, and probably add it to the document so that the others encountering the same issue would feel less confused? Your explanation would be greatly appreciated!
Expectations
case for arrow function
Code of Conduct

[X] I agree to follow Biome's Code of Conduct

The no-redundant-use-strict rule kicks in because JS modules are automatically strict.
CommonJS
If you create a test.cjs (CommonJS) file with the following contents, the rule will not kick in, unless you actually have a redundancy:
// Rule won't kick in
(() => {
    "use strict";
})();

// Rule will kick in
"use strict";
(() => {
    "use strict";
})();

test.cjs:3:5 lint/suspicious/noRedundantUseStrict  FIXABLE  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✖ Redundant use strict directive.

    1 │ "use strict";
    2 │ (() => {
  > 3 │     "use strict";
      │     ^^^^^^^^^^^^^
    4 │ })();

  ℹ This outer use strict directive already enables strict mode.

JS Module
However, when using JS modules, the rule will kick in.
(() => {
    "use strict";
})();

test.mjs:2:5 lint/suspicious/noRedundantUseStrict  FIXABLE  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

  ✖ Redundant use strict directive.

    1 │ (() => {
  > 2 │     "use strict";
      │     ^^^^^^^^^^^^^
    3 │ })();

  ℹ The entire contents of JavaScript modules are automatically in strict mode, with no statement needed to initiate it.

Biome's behavior
Biome seems to consider .js files as JS modules by default.
Documentation
I agree with you that the documents should be clarified.

Unfortunately I don’t

That's the intended behaviour. The greatest majority of the code bases are modern, so we assume that .js files are modules. If not, you have two options:

use a .cjs file
suppress the rule

We will make this rule smarter in the future.

That's the intended behaviour. The greatest majority of the code bases are modern, so we assume that .js files are modules. If not, you have two options:

use a .cjs file
suppress the rule

We will make this rule smarter in the future.

Is there any way I can mark my .js file as a CommonJS file? Why I am asking this because I am working on a plugin for Obsidian which expects a CommonJS main.js file. Having biome detect it as a CommonJS module would be really helpful.

Unfortunately, no. But as I said, soon we will be able to make the rule smarter.
Do you have a package.json?

Yes, I do.

Is this now fixed?


RFD - Start moving services into their own Kubernetes Namespaces

Status
Open for comments 💬

Author(s)
@Adam-D-Lewis

Date Created
29-08-2024

Date Last updated
29-08-2024

Decision deadline
dd-MM-YYY

Start moving services into their own Kubernetes Namespaces
Summary
Services integrated into Nebari often suggest putting the service in it's own namespace when used with Kubernetes (e.g. Rook Ceph, Argo Workflows).  Jupyterhub also has the option to put users in their own namespace.  So far we've kept everything in a single namespace.  The idea being that if someone wants to deploy Nebari into an existing Kubernetes cluster they can be assured that by default everything is deployed in a single namespace.

Benefit

Putting services and potentially users in their own namespaces brings some advantages around controlling resource limits on users.  It also allows us to follow the best practices recommended by the services themselves.
It can also help with some aspects of RBAC permissions.  E.g. All pods in a namespace have all the the permissions granted to the default service account (assuming automountServiceToken: true).  This is a smaller benefit in my mind.  We currently don't use the default service account.
Design Proposal
I propose we move Argo Workflows, Keycloak, Rook-Ceph, Monitoring (Prometheus/Grafana), potentially Jupyterhub each into their own namespaces.  We should ensure the namespace is configurable so users could put everything in a single namespace if they wish, but by default we separate these things out.

Alternatives or approaches considered (if any)

Best practices
Services recommend being put in their own namespace.  It also gives better resource controls on various services and potentially Nebari users.

User impact

Ideally none, there may be some special instructions/manual steps required around upgrading Nebari b/c of these changes.
Unresolved questions

What changes will need to be made to support services in their own namespace?  I think it will be minimal for most services with the possible exception of Juyterhub.

I do like this approach, and I would prefer we opted it as default behavior. We already have control over the services endpoint injection in each helm chart, as well as, the infra resources itself. So, we could have the existing deployment option default to single namespaces.

Services accounts such as the jupyterhub one would possibly require access to the user's namespaces, so that would need to use some level of tagging system or similar if that exists.

Copying over some internal discussion here.

Dharhas Pothina
The earlier issue about doing that was the idea that some orgs wouldn't be able to do a wildcard dns entry.
so it would need be optional.

Vinicius Cerutti
I think the main goal is to separate the resource at the kubernetes level to separate namespaces, the endpoints entries should, in theory, be kept the same. (edited)
my single concern is with deployments on existing infra, where the number of namespaces and their access might be limited. (edited)

Adam Lewis
Yes, that's what I'm proposing Vinicius Cerutti. My understanding is the single namespace design originally was intended for orgs wanting to do a deployment into an existing k8s cluster.  At least they could have some peace of mind that everything is going into a single namespace.  However, it doesn't seem like that is very common at all.  Do we know of anyone doing this currently?  It seems like not the best to ignore best practices for a scenario which isn't really happening.

Adam Lewis
We could continue to support single namespace deployments, but just make the default multi namespace. (edited)

Adam Lewis
Most of the CRDs, cluster roles are for services like Prometheus, Argo, etc.  I think making it easier for orgs doing existing k8s Nebari deployments to use their own e.g. Prometheus deployment, Argo Deployment, etc. with Nebari if they have one would be a better way. (edited)

Chuck McAndrew
I am in favor, but we should expose ways to set the namespaces for each service  so that if an org wants everything in one, we can do that

Adam Lewis
Yeah, I think keeping namespace configurable is totally reasonable.

On meeting today there was also a mention of whether this has an interaction with:

enabling per-user subdomains in JupyterHub https://jupyterhub.readthedocs.io/en/stable/explanation/websecurity.html#enable-user-subdomains
c.KubeSpawner.enable_user_namespaces https://discourse.jupyter.org/t/how-to-create-their-own-namespace-for-each-user/8039

There was also a comment that the namespaces should be deployment specific so that we could have multiple Nebari deployments on a single cluster.

Issue opened for this - https://github.com/nebari-dev/nebari/issues/2826


Log hpx threads on forced shutdown

When an HPX process hangs (usually because some hpx thread hasn't terminated properly), the process is often killed with a termination signal.
In such cases, logging information about any alive hpx threads would really help me in debugging, especially when an issue is hard to reproduce locally.
@hkaiser I can certainly add this if you also think it makes sense, only could you please point me to where exit signals are handled? Thanks!

@Pansysk75 Thanks! The termination handlers are here: https://github.com/STEllAR-GROUP/hpx/blob/bc993dd15b9252908e0b30b9ed5efd830cf99909/libs/core/runtime_local/src/runtime_local.cpp#L65-L174

Issue closed by merging of PR https://github.com/STEllAR-GROUP/hpx/pull/6340


Focus area proposal: Economic value

Goal: To identify metrics that help to measure how various stakeholders drive economic value from open source.
One already released metric is labor investment that we can move to this focus area. Similarly, we can brainstorm for other metrics like different revenue streams and opensourcing.
Any other ideas for metrics for the proposed focus group are most welcome.
Note: I got the concept of open sourcing from the following article.
Ågerfalk, P. and Fitzgerald, B. 2008. Outsourcing to an Unknown Workforce: Exploring Opensourcing as a Global Sourcing Strategy. MIS Quarterly. 32(2) 385-409.

Closing this in favor of #93


Adjust and create new guards and routers for new pages

Duplicate issue: Routing already completed by someone else Page names should be adjusted to correct names
#115
Guards are incomplete.


Researching Database technologies

Vertailuun mahdolliset database teknologiat.
Kuinka standardi SQL:lää?
Neo4j
MySQL Vektori laajennus

Research 'swappability' of different technologies: How similar are the instructions/language, can solutions be swapped for another without major reworks to the code?

Database/env. can be containerized from the start.

database research.pdf
Result:
Neo4j seems to have a good chance for providing all the utilities from start to end from prototyping point of view. While MySQL vector extension might not have needed features after midway of the project.


Confirm footer information, attach samples

I did ask but Ms Dudu said she wants to discuss it with others first before she can confirm or add anything

Sample of footer

Will put together a sample, for us to work around


Exception when running demo.initialize()

File ".../env/local/lib/python2.7/site-packages/gcloud/datastore/demo/__init__.py", line 26, in initialize
    datastore.set_default_dataset_id(DATASET_ID)
AttributeError: 'module' object has no attribute 'set_default_dataset_id'

When running with:
GCLOUD_TESTS_PROJECT_ID=X \
GCLOUD_TESTS_DATASET_ID=X \
GOOGLE_APPLICATION_CREDENTIALS=foobar.json \
python -m gcloud.datastore.demo

on version 0.7.0

@clement Thanks a lot for filing! You inspired to file #1049
Looking into it. It's still broken at HEAD (I just checked).

@clement This won't be fixed until we cut a new release. But that may be very soon. (/cc @tseaver)


Library Request: scipy (Explicit Addition)

Expected Behavior
Upgrades scipy to version >1.6.0

SciPy (pronounced “Sigh Pie”) is a Python-based ecosystem of open-source software for mathematics, science, and engineering.

Actual Behavior
We do support scipy but we should explicitly add it via pip install, and upgrade to >1.6 to include the following bug fix.
Notes
This library would be used for pandas transformation in user code, not replace pandas in Lean PandasConverter.
Checklist

[x] I have completely filled out this template
[x] I have confirmed that this issue exists on the current master branch
[x] I have confirmed that this is not a duplicate issue by searching issues

Any update on this? The new version is supposed to fix this issue! https://github.com/scipy/scipy/issues/11403

Any update on this? The new version is supposed to fix this issue! https://github.com/scipy/scipy/issues/11403

Following up here as well. Is this an easy add?


Log Generation in Batch and Stream Mode to AWS S3

AWS metrics updated for all modes

new pull request


Adding examples for dataflow-streaming

First batch of comments

Did the first version compile? If so, there must be pubsub-v1beta1 jar somewhere in your classpath.

LGTM, will merge


asdf: 3.3.2 -> 3.3.3

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/asdf/versions.
meta.description for asdf is: '"Standard software-system definition library for Common Lisp"'.
meta.homepage for asdf is: '""

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
/bin/sh passed the binary check.
1 of 1 passed binary check by having a zero exit code.
0 of 1 passed binary check by having the new version present in output.
found 3.3.3 with grep in /nix/store/bizskr92zsgsz5c8jffh4kba10c8y20m-asdf-3.3.3
directory tree listing: https://gist.github.com/150e99c2e379dd125c192d40b597d81e
du listing: https://gist.github.com/8482a5d896d2d0d0e28c481f01a18df0

Rebuild report (if merged into master) (click to expand)

54 total rebuild path(s)
52 package rebuild(s)
52 x86_64-linux rebuild(s)
1 i686-linux rebuild(s)
0 x86_64-darwin rebuild(s)
1 aarch64-linux rebuild(s)
First fifty rebuilds by attrpath
asdf
lispPackages.caveman
lispPackages.cffi
lispPackages.cffi-grovel
lispPackages.cffi-toolchain
lispPackages.circular-streams
lispPackages.cl-async
lispPackages.cl-async-base
lispPackages.cl-async-repl
lispPackages.cl-async-ssl
lispPackages.cl-async-util
lispPackages.cl-fuse
lispPackages.cl-fuse-meta-fs
lispPackages.cl-l10n
lispPackages.cl-libuv
lispPackages.cl-mysql
lispPackages.cl-pdf
lispPackages.cl-project
lispPackages.cl-smtp
lispPackages.cl-test-more
lispPackages.cl-typesetting
lispPackages.cl_plus_ssl
lispPackages.clack
lispPackages.clack-handler-hunchentoot
lispPackages.clack-test
lispPackages.clack-v1-compat
lispPackages.dbd-mysql
lispPackages.dbd-sqlite3
lispPackages.dexador
lispPackages.drakma
lispPackages.fast-http
lispPackages.fast-io
lispPackages.http-body
lispPackages.hu_dot_dwim_dot_asdf
lispPackages.hu_dot_dwim_dot_stefil
lispPackages.hunchentoot
lispPackages.iolib
lispPackages.iolib_dot_grovel
lispPackages.jonathan
lispPackages.lack-middleware-backtrace
lispPackages.lev
lispPackages.lfarm-ssl
lispPackages.prove
lispPackages.query-fs
lispPackages.quicklisp-to-nix
lispPackages.quicklisp-to-nix-system-info
lispPackages.smart-buffer
lispPackages.sqlite
lispPackages.static-vectors
lispPackages.uiop

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/bizskr92zsgsz5c8jffh4kba10c8y20m-asdf-3.3.3 \
  --option binary-caches 'https://cache.nixos.org/ https://r-ryantm.cachix.org/' \
  --option trusted-public-keys '
  r-ryantm.cachix.org-1:gkUbLkouDAyvBdpBX0JOdIiD2/DP1ldF3Z3Y6Gqcc4c=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(r-ryantm's Cachix cache is only trusted for this store-path realization.)
Or, build yourself:
nix-build -A asdf https://github.com/r-ryantm/nixpkgs/archive/bd31f06d8758306e862ee83458559010d8db4fec.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/bizskr92zsgsz5c8jffh4kba10c8y20m-asdf-3.3.3
ls -la /nix/store/bizskr92zsgsz5c8jffh4kba10c8y20m-asdf-3.3.3/bin

cc @7c6f434c for testing.

@jonringer Thanks


chore: Add security and license scanning with FOSSA

Continuation of the #722 - now that we assessed and landed on FOSSA
Now we are ready to merge the scan -- it's scanning as it should, although it reported the following errors already:
Using project name: `https://github.com/TBD54566975/ftl`
Using revision: `0d3b05517292a2ec0cc9824d6a12894a4dbee0c5`

[ERROR]

  ========================================================================
  Tested Following Project:
  ========================================================================

  Project Title: https://github.com/TBD54566975/ftl
  Project Revision: 0d3b05517292a2ec0cc9824d6a12894a4dbee0c5
  Project Visibility: private
  Project Targets:
  - gomod: [buildengine/testdata/projects/alpha/go.mod]
  - gomod: [buildengine/testdata/projects/another/go.mod]
  - gomod: [buildengine/testdata/projects/external/go.mod]
  - gomod: [buildengine/testdata/projects/lib/go.mod]
  - gomod: [buildengine/testdata/projects/other/go.mod]
  - gomod: [go-runtime/compile/build-template/go.mod]
  - gomod: [go-runtime/compile/external-module-template/go.mod]
  - gomod: [go-runtime/compile/testdata/duplicateverbs/go.mod]
  - gomod: [go-runtime/compile/testdata/failing/go.mod]
  - gomod: [go-runtime/compile/testdata/one/go.mod]
  - gomod: [go-runtime/compile/testdata/two/go.mod]
  - gomod: [go-runtime/scaffolding/go.mod]
  - gomod: [go.mod]
  - gomod: [integration/testdata/schema-generate/go.mod]
  - gomod: [kotlin-runtime/external-module-template/go.mod]
  - gomod: [kotlin-runtime/scaffolding/go.mod]
  - maven: [buildengine/testdata/projects/externalkotlin/pom.xml]
  - maven: [buildengine/testdata/projects/libkotlin/pom.xml]
  - maven: [kotlin-runtime/ftl-runtime/pom.xml]
  - maven: [kotlin-runtime/scaffolding/{{ .Name | lower }}/pom.xml]
  - npm: [extensions/vscode/package-lock.json]
  - npm: [frontend/package-lock.json]

  COMPLIANCE ISSUES (Total 10)

  ========================================================================
  Unlicensed Dependency (Total 1)
  ========================================================================
  ⚑ Unlicensed dependency detected in github.com/alecthomas/kong-toml@09a5bdacdc2ae63e5c06164b372de9d7234ae691
  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/4433957

  ========================================================================
  Flagged by Policy (Total 9)
  ========================================================================
  ⚑ LGPL-3.0-only license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203514

  ⚑ EPL-1.0 license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203513

  ⚑ LGPL-2.1-only license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203512

  ⚑ EPL-1.0 license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203517

  ⚑ LGPL-3.0-only license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203516

  ⚑ LGPL-2.1-only license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/5203515

  ⚑ MPL-2.0 license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/4433963

  ⚑ MPL-2.0 license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/4433964

  ⚑ MPL-1.1 license detected in<EMAIL_ADDRESS>  More information: https://app.fossa.com/projects/custom%2B588%2Fgithub.com%2FTBD54566975%2Fftl/refs/branch/master/0d3b05517292a2ec0cc9824d6a12894a4dbee0c5/issues/licensing/4433958

[ERROR] ----------
  An issue occurred

  >>> Relevant errors

    Error

      The scan has revealed issues. Number of issues found: 10

Added a license to kong-toml.

⚑ Unlicensed dependency detected in github.com/alecthomas/kong-toml@09a5bdacdc2ae63e5c06164b372de9d7234ae691

Is there a dependency lock concept in go?
For some reason the scan is running against an old commit:
 ⚑ Unlicensed dependency detected in github.com/alecthomas/kong-toml@09a5bdacdc2ae63e5c06164b372de9d7234ae691
If so, I believe the next step is to update it...
In any case, I believe this PR is now good to be merged, and once in main you will have some red checks until all the license/security issues are solved!
If you agree, feel free to go ahead and merge!

Added a license to kong-toml.

Is there a dependency lock concept in go?
For some reason the scan is running against an old commit:  ⚑ Unlicensed dependency detected in github.com/alecthomas/kong-toml@09a5bdacdc2ae63e5c06164b372de9d7234ae691
If so, I believe the next step is to update it...
In any case, I believe this PR is now good to be merged, and once in main you will have some red checks until all the license/security issues are solved!
If you agree, feel free to go ahead and merge!

@leordev thank you! what do you mean by red checks? will these interfere with other GH action pipelines until resolved?

@leordev thank you! what do you mean by red checks? will these interfere with other GH action pipelines until resolved?

They will not interfere with any other GH action pipelines, you will just have a red cross on the security/license check until its solved.

Is there a dependency lock concept in go?

Ah I forgot to tag a release - done now with v0.2.0


format the configuration about openstack

format the configuration of openstack cloud provider used by kubernetes

This change is

I agree with @tengqm! While the table looks much better the footnotes at the bottom are now no longer footnotes and how they referenced table entries is no longer there.

This was correctly addressed by https://github.com/kubernetes/website/pull/6138

I know, thanks your review.


[TestProxy] Cannot locate recordings when debugging with VSCode

When I run a test-proxy test via the terminal, it puts the recording in the correct place, regardless of my CWD and seemingly regardless of whether I have TEST_PROXY_FOLDER set. For example, if I run a search test, it correctly places the recording in:
{repoRoot}/sdk/search/azure-search-documents/tests/recordings/
However, because this fails on playback, I need to be able to debug into the process to figure out what is going wrong. However, when I run the test using the pytest testing extension for VSCode, test proxy places the recordings in the wrong location when live and cannot locate them on playback.
To make the debugger see the environment variables, I have to use a .env file. In it I specify both TEST_PROXY_FOLDER and AZURE_TEST_RUN_LIVE=true. While it picks up the "run live" signal, it places the recordings at the repo root, regardless of what I put in TEST_PROXY_FOLDER (it doesn't seem to care about it at all).
{repoRoot}/tests/recordings/
However, when I switch AZURE_TEST_RUN_LIVE=false in the .env file, it tries to play back but I have no idea where it is looking for the recording but it can't find it.

Additionally, this pops up any time I try to run or debug tests in VSCode via the debug pane:

This does not happen when running regular tests, but I can't tell what is actually trying to load this obviously bad URL.

And here's the debug log from the test run
============================= test session starts =============================
platform win32 -- Python 3.9.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: f:\repos\azure-sdk-for-python\sdk\search\azure-search-documents, configfile: ..\..\..\setup.cfg
plugins: asyncio-0.16.0, cov-3.0.0
collected 1 item

tests\test_search_indexer_client_live.py F                               [100%]

================================== FAILURES ===================================
__________ TestSearchIndexersClient.test_delete_indexer_if_unchanged __________

test_id = 'tests/recordings/test_search_indexer_client_live.pyTestSearchIndexersClienttest_delete_indexer_if_unchanged'
    def start_record_or_playback(test_id):
        # type: (str) -> Tuple(str, dict)
        """Sends a request to begin recording or playing back the provided test.

        This returns a tuple, (a, b), where a is the recording ID of the test and b is the `variables` dictionary that maps
        test variables to values.
                # This ensures that a user must _very specifically say they want caching_ on an item and all parents.
                if not self._use_cache and child_is_cached:
                    raise Exception(
                        """Preparer exception for test {}:\n Child preparers are cached, but parent {} is not.
    You must specify use_cache=True in the preparer decorator""".format(
                            test_class_instance, self.__class__.__name__
                        )
                    )
                self._use_cache |= child_is_cached
                _logger.debug("Child cache status for %s: %s", self.__class__.__name__, child_is_cached)

                # We must use a cache_key that includes our parents, so that we get a cached stack
                # matching the desired resource stack. (e.g. if parent resource has specific settings)
                try:
                    aggregate_cache_key = (self._cache_key, kwargs["__aggregate_cache_key"])
                except KeyError:  # If we're at the root of the cache stack, start with our own key.
                    aggregate_cache_key = self._cache_key
                kwargs["__aggregate_cache_key"] = aggregate_cache_key
                self._aggregate_cache_key = aggregate_cache_key
                _logger.debug("Aggregate cache key: %s", aggregate_cache_key)

                # If cache is enabled, and the cached resource exists, use it, otherwise create and store.
                if self._use_cache and aggregate_cache_key in AbstractPreparer._resource_cache:
                    _logger.debug("Using cached resource for %s", self.__class__.__name__)
                    with self._cache_lock:
                        resource_name, kwargs, _ = AbstractPreparer._resource_cache[aggregate_cache_key]
                else:
                    resource_name, kwargs = self._prepare_create_resource(test_class_instance, **kwargs)

                if self._use_cache:
                    with self._cache_lock:
                        if aggregate_cache_key not in AbstractPreparer._resource_cache:
                            _logger.debug("Storing cached resource for %s", self.__class__.__name__)
                            AbstractPreparer._resource_cache[aggregate_cache_key] = AbstractPreparer.ResourceCacheEntry(
                                resource_name, kwargs, self
                            )

                if test_class_instance.is_live:
                    # Adding this for new proxy testcase
                    if hasattr(test_class_instance, "scrubber"):
                        test_class_instance.scrubber.register_name_pair(resource_name, self.moniker)
                    else:
                        _logger.info(
                            "This test class instance has no scrubber, so the AbstractPreparer will not scrub any values "
                            "in recordings."

        This returns a tuple, (a, b), where a is the recording ID of the test and b is the `variables` dictionary that maps
        test variables to values. If no variable dictionary was stored when the test was recorded, b is an empty dictionary.
        """
        head_commit = subprocess.check_output(["git", "rev-parse", "HEAD"])
        current_sha = head_commit.decode("utf-8").strip()
        variables = {}  # this stores a dictionary of test variable values that could have been stored with a recording

        if is_live():
            result = requests.post(
                RECORDING_START_URL,
                headers={"x-recording-file": test_id, "x-recording-sha": current_sha},
            )
            recording_id = result.headers["x-recording-id"]
        else:
            result = requests.post(
                PLAYBACK_START_URL,
                headers={"x-recording-file": test_id, "x-recording-sha": current_sha},
            )
            try:
                recording_id = result.headers["x-recording-id"]
            except KeyError:
>               raise ValueError("No recording file found for {}".format(test_id))
E               ValueError: No recording file found for tests/recordings/test_search_indexer_client_live.pyTestSearchIndexersClienttest_delete_indexer_if_unchanged
..\..\..\tools\azure-sdk-tools\devtools_testutils\proxy_testcase.py:98: ValueError
---------------------------- Captured stdout setup ----------------------------
ambitious_azsdk_test_proxy
----------------------------- Captured log setup ------------------------------
INFO     root:proxy_docker_startup.py:116 Starting the test proxy container...
-------------------------- Captured stdout teardown ---------------------------
ambitious_azsdk_test_proxy
---------------------------- Captured log teardown ----------------------------
INFO     root:proxy_docker_startup.py:154 Stopping the test proxy container...
- generated xml file: C:\Users\travi\AppData\Local\Temp\tmp-43028yxVc8eYLe9x6.xml -
=========================== short test summary info ===========================
FAILED tests/test_search_indexer_client_live.py::TestSearchIndexersClient::test_delete_indexer_if_unchanged
============================== 1 failed in 6.74s ==============================

It's also worth noting that the test-proxy XML file referenced (C:\Users\travi\AppData\Local\Temp\tmp-43028yxVc8eYLe9x6.xml) does NOT exist.

Part of the problem is that proxy_testcase.py is swallowing the test-proxy error and rethrowing a more vague and unhelpful error. The surfaced error says "No recording file found for {test_id}". But the REAL error is this:
"System.IO.FileNotFoundException: Could not find file '/srv/testproxy/tests/recordings/test_search_indexer_client_live.pyTestSearchIndexersClienttest_delete_indexer_if_unchanged.json'.\nFile name: '/srv/testproxy/tests/recordings/test_search_indexer_client_live.pyTestSearchIndexersClienttest_delete_indexer_if_unchanged.json'

So the path that test-proxy is looking for is /srv/testproxy/tests/recordings/test_search_indexer_client_live.pyTestSearchIndexersClienttest_delete_indexer_if_unchanged.json
Again, why it looks here when TEST_PROXY_FOLDER is set in the .env file does not make sense. Why it looks in two DIFFERENT places depending on whether you are running live or not is even more baffling.
cc/ @mccoyp @scbedd

@mccoyp can you please take at this item? I'll jump in, but given that it's fairly specific to python I want you take a look first.

This bug is coming from how pytest populates the PYTEST_CURRENT_TEST environment variable. For whatever reason, that environment variable usually provides a repo root-relative path to the test, but when debugging in VS Code (in live or playback mode), provides a package root-relative path.
For example:

When running a pytest session for a particular test from a terminal (non-debug), PYTEST_CURRENT_TEST is
'sdk/search/azure-search-documents/tests/test_search_indexer_client_live.py::TestSearchIndexersClient::test_delete_indexer_if_unchanged (call)'.

When debugging the same test via VS Code, PYTEST_CURRENT_TEST is
'tests/test_search_indexer_client_live.py::TestSearchIndexersClient::test_delete_indexer_if_unchanged (call).

Tests pass in live, debug mode when AZURE_SKIP_LIVE_RECORDING is true or false. If recording isn't skipped, test recordings are placed in a tests directory created at the root of the repository.

That's interesting. My observation was when in record, the recording is placed at the package root, but when I tried playback it looked for the test in the test-proxy server folder, not anywhere in the repo.

@tjprescott Now that #22508 has been merged, let me know if this issue persists. If you're now able to load the recording files in debug mode, feel free to close this!

Cool, yes it now seems that the recording is placed in the right location and it does find it. While I run into other issues, at least this one can be closed!


Adds bitset

Adds a somewhat c++ standard conforming bitset. Some features have not been tested, and the formatting is probably incorrect.

Do you have a .clang-format or something similar for formatting?

We have a style guide.

We don't indent namespaces, and the tests aren't indented properly still

Could y'all see if this can be merged?

I'm not precisely sure what you mean by "writing code that can easily migrate to the standard bitset," but the other changes have been implemented.

what I mean is that c++ already provides a <bitset> so it might be valuable to match the interface that one provides so that one can switch between it and frigg easily


Solve syntax errors when training Colbert

Hello @okhat
I want to train Colbert on a specific dataset, but when I run the following code:
CUDA_VISIBLE_DEVICES="0" \
  python -m /lib/site-packages/torch.distributed.launch --nproc_per_node=1 -m /lib/site-packages/colbert.train \
  --amp --doc_maxlen 180 --mask-punctuation --bsize 32 --accum 1 \
  --triples /Dataset/Triple_for_train/colbert_train_triple.tsv

or
CUDA_VISIBLE_DEVICES="0" \
  python -m torch.distributed.launch --nproc_per_node=1 -m colbert.train \
  --amp --doc_maxlen 180 --mask-punctuation --bsize 32 --accum 1 \
  --triples /Dataset/Triple_for_train/colbert_train_triple.tsv

I get this error:
Error:
Input In [21]
    python -m /lib/site-packages/torch.distributed.launch --nproc_per_node=1 -m /lib/site-packages/colbert.train \
    ^
SyntaxError: cannot assign to operator

How can I solve this problem?
Thank you.

I'd navigate to the directory that contains the colbert/ folder and then run:
CUDA_VISIBLE_DEVICES="0" \
  python -m torch.distributed.launch --nproc_per_node=1 -m colbert.train \
  --amp --doc_maxlen 180 --mask-punctuation --bsize 32 --accum 1 \
  --triples /Dataset/Triple_for_train/colbert_train_triple.tsv

Does this not work for you?

No, It`s not work
and I get this error:
python -m torch.distributed.launch --nproc_per_node=1 -m colbert.train \
^
SyntaxError: invalid syntax

I use python 3.8 and jupyter notebook

Oh, I see. You're running this inside a Jupyter notebook.
In that case, just add ! before the command.
!python -m torch.distributed.launch --nproc_per_node .....

Thank you Omar
I had forgotten !
I ran this part of the command (CUDA_VISIBLE_DEVICES="0" )in one cell and the rest of the command in another cell
!python -m torch.distributed.launch --nproc_per_node=1 -m colbert.train \
--amp --doc_maxlen 180 --mask-punctuation --bsize 32 --accum 1 \
--triples /Dataset/Triple_for_train/colbert_train_triple.tsv

But now I have the following error:
RuntimeError: Default process group has not been initialized, please make sure to call init_process_group.
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 10180) of binary: C:\ProgramData\Anaconda3\python.exe

I tried to initialize the init_process_group with the following command:
dist.init_process_group('gloo', init_method='file:///C:/init/some_file', rank=0, world_size=1)

But the error was not fixed.
Do you have any solution?
Thanks for your reply

--amp --doc_maxlen 180 --mask-punctuation --bsize 32 --accum 1 \
--triples /Dataset/Triple_for_train/colbert_train_triple.tsv

This should work, since you're using one GPU.

Thank you for your reply
The problem was solved.

Hi again @okhat
I tried to train the model using my dataset, but in 5 minutes, the model will be trained. I use a Persian dataset. Do you think this is a reasonable amount of time to train the model?
My train file contains 18,000 queries, for each query there is a positive passage and some negative passages. Since the number of negative passages per query is more than one, I have put the negative passages of each query in a list. Is it okay to use the list in this case?
RAM: 54GB
GPU: RTX 2080 with 12 GB


Support custom app name for prometheus-operator-crd when installing CRDs

Closes https://github.com/giantswarm/giantswarm/issues/21262

Solved by https://github.com/giantswarm/prometheus-operator-app/pull/178


cuDevicePrimaryCtxRetain: CUDA_ERROR_INVALID_DEVICE ... Failed to create session

Hello everyone, when I execute this code :
import tensorflow as tf
hello = tf.constant('hi,tensorflow')
sess = tf.Session()

I get each time the same error :
2018-06-29 15:14:01.587077: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow bhis actinary was not compiled to use: AVX2 FMA
2018-06-29 15:14:01.694678: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:898] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2018-06-29 15:14:01.695507: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1356] Found device 0 with properties:
name: Quadro P400 major: 6 minor: 1 memoryClockRate(GHz): 1.2525
pciBusID: 0000:05:00.0
totalMemory: 1.95GiB freeMemory: 1.92GiB
2018-06-29 15:14:01.715820: E tensorflow/core/common_runtime/direct_session.cc:154] Internal: failed initializing StreamExecutor for CUDA device ordinal 1: Internal: failed call to cuDevicePrimaryCtxRetain: CUDA_ERROR_INVALID_DEVICE
Traceback (most recent call last):
  File "/home/mounir/PycharmProjects/ssd_keras-master/ssd512_inference.py", line 6, in <module>
    sess = tf.Session()
  File "/home/mounir/anaconda3/envs/gpukeras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1560, in __init__
    super(Session, self).__init__(target, graph, config=config)
  File "/home/mounir/anaconda3/envs/gpukeras/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 633, in __init__
    self._session = tf_session.TF_NewSession(self._graph._c_graph, opts)
tensorflow.python.framework.errors_impl.InternalError: Failed to create session.

I have two GPUs in my computer, the P400 is the most powerful one.
I tried to troubleshoot it by adding to .bashrc export CUDA_VISIBLE_DEVICES with alternatively these values =0, =1, =0,1, but it outputs always the same error
Have I written custom code :
import tensorflow as tf
hello = tf.constant('hi,tensorflow')
sess = tf.Session()

OS Platform and Distribution :  Linux, Ubuntu 18.04
TensorFlow installed from : PyPI
TensorFlow version 1.8.0
Bazel version Seems not installed
CUDA/cuDNN version Cudatoolkit installed within the virtual env is 9.0 and Cuda installed within the system is 9.1
GPU model and memory P400 Quadro, 2 GB
Exact command to reproduce
import tensorflow as tf
hello = tf.constant('hi,tensorflow')
sess = tf.Session()

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

I'm not sure what you mean by "Cudatoolkit installed within the virtual env is 9.0". I do not think it's possible to install CUDA in a virtualenv. You should only have one CUDA version, and currently the TensorFlow nightlies only work with Cuda 9.0. Try replacing the system CUDA with 9.0 if possible, or use docker.

It has been 14 days with no activity and the awaiting response label was assigned. Is this still an issue?

Nagging Assignee @zheng-xq: It has been 16 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @zheng-xq: It has been 31 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Hello, I abandoned the double GPU setup. I kept only the Quadro P400, and I am intending to reinstall everything from scratch. This problem was probably, essentially, due to a bad connection between Cuda and the GPU version of Tensorflow. I think that it is suitable to close the issue as I may have understood what the source of the problem was.


localStorage polyfill doesn't work on Safari private browsing (window.localStorage cannot be overwritten)

Safari private browsing disables localStorage but also makes window.localStorage readonly, inconveniently. What's the best workaround for this?

I will take a look om the problem one of the next days. But for now:
As I recall, when using safe browsing, a try catch around the "setItem" and "getItem" will catch an error. In such situations one could fallback to using one of the other solutions, such as the FlashStorage.

I ran the tests in incognito mode in the newest versions of Firefox, Chrome and Opera and no problems seems to occur there. I'll try to get access to a Windows machine to tests the Safari private browsing.

Yeah AFAIK this is a specifically Safari problem. I have notes on it somewhere but the jist of it is that each of the browsers handles private sessions differently and that Safari's (decidedly un-private) method is to make localStorage available but readonly.
A Googling turned this up: https://blog.whatwg.org/this-week-in-html-5-episode-30

Sorry to revamp an old issue, but what @christopherliu is saying is that you cannot overwrite localStorage when Safari is in private mode.
https://github.com/mortzdk/localStorage/blob/master/dist/localStorage.js#L694
Any plans on working around this?

I will take a look at the problem, but I must postpone the matter till after the 21. of June, since I'm in the process of writing my Master's Thesis.

Was passing to a problem like this one described and found a module that deals using this: https://raw.githubusercontent.com/ripeworks/local-storage-fallback/master/src/hasStorage.js

The same technique I use in version 3.0, which is on its way :)

Hi @mortzdk  , thanks for the good work, Just wondering if version 3.0 is out and/or safari issue in private browsing has been fixed?.
Thank you again

Hi @eduzol, sorry for not making the v3.0 branch public. It is now public as v3.0 i.e. it is not released yet.
Feel free to do some testing :)

@mortzdk Whats the release schedule?
How does localStorage persist information in Incognito mode?


[Rogue] Update Gear/Talents/APLs; Fix for zombie bleed ticks

Number of changed files is spooky but a lot of it is deleting old APLs and Gearsets.
Also includes updates to Venomous Wounds to prevent zombie ticks of Rupture and Garrote, and Gouge for use in Combat's APL.
This PR primarily focused on Combat and Assassination. Subtlety I need more time to fine-tune, but want to get the other two specs out the door so it can be validated by TCers.

You should look into SpellMods and refactor the rogue to use them. Maybe not in this PR but should be soon


ButtonGroup Component

Hey there,
I needed a ButtonGroup Component (just the visual part) so I created this one, based purely on css.
It must contain Button elements.
ButtonGroup.svelte

<slot></slot>

You can see it in action here
You are also more than welcome to use it in any way you see fit or include it in the lib, with or without modifications.

Hi @sourcegr,
thanks a lot, I immediately found a feature in Safari, I have to reset the margin user agent stylesheet on the button. And I'll include it in the lib.

👍 btw, the popover is a very nice addition ;) It will come in handy!


Better search

What I'm trying to achieve
I would like to have more complex search queries.
If I want to search for 2 names of products for example I have to run two queries.
Describe a proposed solution
Can we add support for regex strings in search?

What's the use case for such search? Customer storefront/dashboar?

You can also access saleor from other apps, for that use-case it's useful to have more complex search queries when you want to get specific products.

There was some research done on our side and we decided that we will come back to search later - probably in Saleor 4.0. So we are speaking about a year from now.
If you need more complicated requests right now there are 2 options:

filter products on client-side
create an app for product data synchronization and sync with elasticsearch

@krzysztofwolski Are there any docs on how to do the second option you mentioned?

@gax97 https://docs.saleor.io/docs/3.0/developer/extending and section apps.
As starting point for building apps you can use my personal boilerplate: https://github.com/krzysztofwolski/saleor-best-app


what happens in case of enforced=True with out any begin time ?

Before the stop time nothing will happen. At or after the stop moment the instance will be stopped if it is running, and stopped again if it is started "manually"

Thanks @arieleeuw


Allow multiple app instances

This might be problematic... See here especially the last comment...

Ok. That is why atom does comment#3

so any solution with multiple edit windows so has to go with tabs...

... yeah, tabs are a good idea! I'll add this feature as a new issue!

Closing because not possible or very tricky. #38 as possible solution.


Don't fetch data from offline backends.

I experienced that having a CLUSTER_NODE offline rendered Graphite
completely unresponsive, and found out that this was due to an uncaught
exception in HTTPConnectionWithTimeout.connect().
In addition to this, render/datalib.py always iterated through all
CLUSTER_NODEs, without considering their 'available' property, i.e. if
they'd been marked as offline.
render/datalib.py:
When fetching remote data, bypass remote stores that aren't available.
remote_storage.py:
Also mark stores unavailable if requests to them fail on network or
transport level.

All those questions make good sense. I'll get back with answers as soon as I can.
My problem is that I haven't found a way to reliably reproduce this problem in my test setup – only in my production setup, which has >50 cluster backends, and in which outages disturb my colleagues.

You might also be interested in https://github.com/criteo-forks/graphite-web/commit/479958f2826c286e15f6d9182466b72aec75b7fd that I haven't had the time to send here.

Is finally merged mine. Is this one still relevant ?

I think no. @nyerup ?

It's definitely outdated by #1810

Closing then. Sorry, @nyerup
Thanks for your contribution, anyway!

All right, makes good sense. Sorry for not getting around to clearing out these questions sooner.
I'll look through #1818, and open a new merge request if I find anything meaningful to contribute.


live rendering img/implementation-plan-graph.dot

live rendering img/implementation-plan-graph.dot with https://pointillism.io
Hope this helps!

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: trevorgrayson sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks, this is a pretty cool timesaver!


➕ Add Dynamic "Last Updated" Functionality to All Legal Pages !

Describe the feature
Hey @Kalivarapubindusree and @Its-Aman-Yadav
I want to add a "Last Updated: " functionality to the legal pages of our website, specifically the Privacy Policy, Terms and Conditions, and Licensing pages.
This functionality will be dynamically adjustable to update monthly and yearly.
Please Refer to the attached image for an example of how the functionality should look

Currently, The Legal Pages Do not have these Updates, Which makes user feel Outdated.
I want to work on this, please assign me.
thankyou
Add ScreenShots
.
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@Its-Aman-Yadav @Kalivarapubindusree I want to work on this. Could you Please assign me


Allow empty returns from functions

Allow returning 0 values from functions.
Reaching the end of a function should return 0 results.

Fixed in v0.12.0


feat(services) - Add Logs

Add a Logs link in service detail view
Like the logs for a container, docker API >= 1.25 allow to get logs for a service (from all containers in all nodes)
See https://docs.docker.com/engine/api/v1.25/#operation/ServiceLogs
I think the portainer log view for container can be reused (displaying stdout and stderr with autorefresh)

would love to see this picked up again now that service logs is stable in Docker 17.05. Would it just be a case of reverting that commit?

We will have a look to merge this in 1.13.x !

@richardgoater seems that we're hitting some bugs in 17.05, we should wait for 17.06.
See #671

thanks @deviantony.


ORC-684: [C++] Make Floating point NaN check more strict

What changes were proposed in this pull request?
ORC-636 added an extra check (part of RecordReaderImp) disabling PPD when DoubleColumnStats min, max or sum is a non-Finite number.
However, the check is unnecessarily broad – the way stats are updated, using just sum for the check would suffice.
Why are the changes needed?
Make NAN check for DoubleStats (used for PPD) more strict
How was this patch tested?
TestPredicateLeaf.cc

Thank you, @pgaref . Is this a performance issue?

Merged to master


5a598359bcbff9b9b90fe3818cfd0dcfb209cb0d

Comments about the best possible price for a while to get a - - # - good ~~~~

Phonepe update is now the next thing you should


public objects and state are cleared when with Delete and Add Row

Hi,
I'm using TypeScript with react-bootstrap-table.
Every time I hit "Add" or "Delete" button, I lose all the objects save in public or state.
ex:
this.data
this.state.data
Any clue why they are lost and how can I maintain them?

any example code for me to check firstly? I can 't debug by your description. Thanks


[Feature] Add Transaction::to_input function

Component
consensus, eips, genesis
Describe the feature you would like
currently input returns a slice even though we know the tx types internally always use Bytes, which makes it expensive to obtain a Bytes instance.
we should add
fn to_input(&self) -> Bytes

for this, this way cloning the input is cheap
cc @tcoratger
Additional context
No response

https://github.com/alloy-rs/alloy/pull/1487 ?

makes sence although this wouldn't work for transactions (non-existing ones) that don't have input. but I guess we can make that assumption


[OCPBUGS-5724]: Update kustomize/kyaml to v0.14.0

The go-yaml package is v2.2.3+, but there was an indirect dependency to kustomize/kyaml.
Also bump kustomize/api.

Failing to build the source code. Waiting for #40 to get merged, which should fix the issues.


VNodeData interface broken

Version
2.2.1
Reproduction Link
https://jsfiddle.net/dn61kvxw/1/
Steps to Reproduce

Observe output

What is expected?
v-model directive expression should be accessible.
What is actually happening?
directives list not found in VNodeData.
Working example with v2.1.10: https://jsfiddle.net/dn61kvxw/2/

Ok, this is because in 2.2 v-model on component is now passed down as part of VNode's data instead of as a directive. So <child v-model="foo"> is compiled into something like this:
h('child', {
  model: {
    value: foo,
    callback: function ($$v) { foo = $$v }
  }
})

Is there any particular reason you need access to the expression?

I have a form implementation where <form-component> provides errors object and <input-*> components can get validation errors from the injected errors object. Validation errors are keyed by the expression.
...
<input-text v-model="attributes.name" />
...
// errors.get('name') would return validation errors for `name` attribute.

I've added the raw expression in 12870be
You should be able to access it as this.$vnode.data.model.expression

The model property is not defined in the VNodeData interface here:
https://github.com/vuejs/vue/blob/982d5a492fb95577217e2dacaa044eabe78a8601/types/vnode.d.ts
Should it be formally added there so we can depend on its presence?


ci: Add release workflow

Changes:

create GitHub release and link to changelog
build and push container image to Docker Hub
build kustomization and upload asset to GitHub release

@hiddeco the changelog will come with the first release, this is just the CI setup.


redission ScheduledFuture can not be execute after definite time

Expected behavior
complete the ScheduledFuture after ten seconds
Actual behavior
ExecutorOptions options = ExecutorOptions.defaults();
    options.taskRetryInterval(10, TimeUnit.MINUTES);
    RScheduledExecutorService executorService = redissonClient.getExecutorService("myExecutor");
    ScheduledFuture<Long> future = executorService.schedule(new TaskTest(), 10, TimeUnit.MINUTES);
    Long result = future.get();
    System.out.println(result);

the schedule task above can not be executed after 10 seconds, the current thread beign blocked
Steps to reproduce or test case
Redis version

Redisson version  3.10.0
Redisson configuration
using spring boot org.redisson.spring.starter.RedissonAutoConfiguration, just configure redis host and port in properties file.

ScheduledFuture future = executorService.schedule(new TaskTest(), 10, TimeUnit.MINUTES);
actual limit time description is TimeUnit.SECONDS

the schedule task above can not be executed after 10 seconds

It's executed after 10 minutes since you defined TimeUnit.MINUTES

It's executed after 10 minutes since you defined TimeUnit.MINUTES

Actually, I had try TimeUnits.SECONDE before, but it did not work.

Did you start RedissonNode or register workers through RExecutorService.registerWorkers method?


Zooming is bad/broken

When you try to zoom-in top right, for instance, it doesn't zoom in top right, it zooms in more in the middle, not precise at all. This library doesn't even achieve the simple pinch the zoom gesture and hence it is useless. I have no idea why this library has 17k stars, because it just doesn't work well. Google's photo library app doesn't have this defect. iOS zooming (Apple has done the work for us unlike Android) doesn't have this defect. When I implemented zooming for a View in Android, it didn't have that defect. I guess I will have to implement my own solution instead of using this library.
I think it should be the top priority of this repo to fix the zooming to work naturally just like Google's built in Photos app or iOS apps. When you zoom in top left, it should zoom in top left where you are zooming, not in the center.

@coolcool1994 This is an open source library, not something that is made by Google.
And it's no one's "top priority" to do the stuff you want.
Have some respect for the effort of the contributors, they did everything for free.
Don't like it - make a pull request and improve it, or open source your own solution.
It would be more helpful than ranting here and insulting the maintainers.

And @coolcool1994 about implementing your own library, feel free, but, reinforcing what friend @Dimezis said, you should show some respect for the work that was done here, and a sincere advice: the library, like as is, works well, so , nothing would cost for you to fork the project, implement the functionality you need, and, since you are already benefiting from an open source project, make a pull request to put these changes to the branch master of it and everyone would be happy.
thanks

and I’m sorry to say, but it’s always that, who does it, leaves it done, who doesn’t do it, go there and put the stick in the buddy who made it and left it ready.

I mean it seems like Yelp app is using this library, because the Yelp has the same flaws. How could Yelp people not contribute!?!? Surprising.
Google Photos library people should contribute honestly. I have my own solution (totally separate project) that doesn't work as smooth as Google Photos library but does work the same way.


[WIP] Create table with FixedSizeList column

Which issue does this PR close?
Closes #10303.
Rationale for this change
See #10303.
To make it possible to create a table with a fixed size list column without resorting to casting.
What changes are included in this PR?
Enable SQL query planner to convert the SQL data type Array(ArrayElemTypeDef::SquareBracket) to a FixedSizeList.
Are these changes tested?
No. Help needed.
Are there any user-facing changes?
No.

@jayzhan211 @alamb can I please get some guidance on how/where to test that this data type mapping from SQL to DF works as expected?

The result of arrow_typeof should be fixedsizelist
query
create table t (a int[3]) as values ([1, 2, 3]), ([4, 5, 6]);

query
select arrow_typeof(a) from t;
----
FixedSizeList

Thanks for your help! Do I add these tests under sqllogictest? If so, is there an existing test_files/*.slt I should add them to? Maybe arrow_typeof.slt?

Thanks for your help! Do I add these tests under sqllogictest? If so, is there an existing test_files/*.slt I should add them to? Maybe arrow_typeof.slt?

Perhaps you can add them to https://github.com/apache/datafusion/blob/main/datafusion/sqllogictest/test_files/ddl.slt
Here are the descriptions:
https://github.com/apache/datafusion/tree/main/datafusion/sqllogictest#readme


Repo: Adding Thanos unit tests to PR checks

Today thanos PR doesn't run unit tests, this adds a new check to make pr passing unit tests automatically. Let me know if it is desirable to have a healthier repo or it slows down dev experience.

[ N/A] I added CHANGELOG entry for this change.
[ X] Change is not relevant to the end user.

Changes

Verification
If this PR passes all checks :)

never mind, ci/circleci is actually running unit tests:


Support zsh completion for subcommand of ws

https://gist.github.com/mitukiii/4954559
https://github.com/ohmyzsh/ohmyzsh/blob/master/plugins/pass/_pass
https://rcmdnk.com/blog/2015/05/15/computer-linux-mac-zsh/
https://masawada.hatenablog.jp/entry/2021/10/08/103000


How do I implement my own NMS into mmdetection?

Hello everyone,
I am trying to implement a rescoring algorithm before NMS for video object detection. I am considering to run this algorithm and after that apply NMS. How do I get the bboxes before NMS during inference?

Or is there a better way ?

You should modify the head of the model that you are using. The inference API is not for development.

You should modify the head of the model that you are using. The inference API is not for development.

Yeah, I see, thanks.  Actually, I figured out how to implement my nms and use it for models, I just need to give an additional input while testing, how should I do that?

multiclass_nms

Has anyone solved it? I wanna output bboxes before NMS, how should I do it?

I did it. Just customize bbox_nms.py . Create your own dummy nms and use it
instead.
JillianWang<EMAIL_ADDRESS>2 Mar 2021 Sal, 09:46 tarihinde
şunu yazdı:

Has anyone solved it? I wanna output bboxes before NMS, how should I do it?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/open-mmlab/mmdetection/issues/4329#issuecomment-788660537,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIW235LPUY2OPMT3DMJ353DTBSCUPANCNFSM4VCECSMA
.

你好，请问一下，您的NWD/mmde/core/post_processing/wasserstein_nms.py中def wasserstein_nms_op这个函数为什么什么内容也没有就pass了，另外def wasserstein_nms和def batched_wasserstein_nms在哪里调用的，调用的时候是不是只选一个调用即可，我的代码是在测试的用的，我划分了train，val, test。


Would like to attend the Selenium Open Space Conference - Ed Manlove :)

Added myself (Ed Manlove) to the list of attendees. Also added a session idea around the future of the python bindings with selenium.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Fixed exceptions in str

Use any_language to avoid exceptions when no fallback is defined
Fix #369

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.

Coverage remained the same at 95.981% when pulling ee154350bf9bda21ee473f3ebe6d41dd82613ab7 on issue-369 into 2b2aaa06784a5925c26b1f1977d74b5f46999677 on develop.


no VSCodeNotifyVisual() cmd on version 0.4.3 ?

i use VSCodeNotifyVisual() cmd for long time.
but this cmd seems not supported on version 0.4.3 ?
is there other cmd to replace it ?
tks.

Use VSCodeNotifyRange and VSCodeNotifyRangePos instead


bump sqld to v0.24.18

@sivukhin, the release process is very simple:

bump sqld (this pr)
push a tag in the form libsql-server-vX.Y.Z


Add option to configure consumer group

This pull request adds the option to configure an eph to a specific consumer group
Fix or Enhancement?

[ ] All tests passed
[ ] Add change to changelog.md

Environment

OS: Write here
Go version: Write here

Thank you! We should have included this initially.
I’m out of pocket right now, but I’ll give this a review later today.


Delete all dataset group resources - new step function

Issue #, if available:
The step functions provided with the sample operations step functions do not provide a way to delete all of the resources created in a dataset group.
Description of changes:
Added a step function to delete all of the resources associated with a given dataset group.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Code changes Approved.
Next Steps: Refactor instructions & git folder hierarchy.


Create ssh enabled images for use in AppService

The changes needed for .Net are
https://github.com/Azure/azure-functions-docker/commit/c6beee8c5a38bd3f1d44390a6bb6de72864f5d34
(For node we might need to launch the node process in debug mode additionally)
We will need to have separate ssh enabled images ( dotnet , node , python to begin with) to avoid security issue with having public username/password in the dockerfile. When used in appservice this is ok since the port is only exposed through kudu which requires auth.
The images will be named with -xdebug as part of their name (Ex: appsvc/php:5.6.38-apache-xdebug_1810291954). When remote debugging is enabled on a site , LWAS will pick the -xdebug image instead of regular image.
Matching launch.json config to use with vs code
{
"name": ".NET Core SSH Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickRemoteProcess}",
"pipeTransport": {
"pipeProgram": "ssh",
"pipeArgs": [ "-T"<EMAIL_ADDRESS>"-p", "PORT_NUMBER", "-i", "PATH_TO_SSHPRIVATEKEY" ],
"debuggerPath": "/root/vsdbg/vsdbg",
"pipeCwd": "${workspaceRoot}",
"quoteArgs": true
}
}
debugger path is preset based on dockerfile debugger installation location.
PORT_NUMBER is from
az webapp remote-connection create -g RESOURCE_GROUP -n APP_NAME -p PORT_NUMBER

a couple of questions Bala.
This debugger is just the dotnet debugger, correct? Is there a different configuration for node and python?
The bit about xdebug is only for php, right? so we don't have to do that since we don't have php image.

Correct. This is just for dotnet. Haven't investigated what debugger we need for python and node.cc @dariagrigoriu
For now it would be good to have the basic dockerfiles structure created for all languages and we can add the required debuggers into the right dockerfile whenever we identify them.
The idea is that even in appservice we won't always use the (debugger installed, ssh enabled) images automatically. The right image will be picked by LWAS depending on the value of remotedebugging in the site config. (this is another pending workitem). The logic is based on whether or not the image name includes xdebug. This is what happens with php blessed images on app service right now.
Until we complete the LWAS piece of work, customers can modify LinuxFxVersion manually with the xdebug images after enabling remotedebugging.


320 update slotted svg code snippets

Summary of the changes
removed abbreviated SlottedSVG and svg from code snippets in components pages. Added functioning code so user can copy working code and not have to manipulate it to make it function
Checklist

[ ] I have manually accessibility tested any changes, if relevant.
[ ] I have raised this pull request against the develop branch

Thanks for the feedback and sorry for the little errors!.All resolved hopefully.

It looks like the updates have been added to the wrong commit. A quick fix is to get all the latest updates from main repo develop into your forked repo (using rebase), create a new branch off forked develop, copy your changes across, add it to a new commit message and then create a new PR.


Deleting deprecated/unused classes/resources

Now that action area is turned on in this branch, we can kill off the deprecated class and remove even more unused resources :D

hmm for some reason the commit I made from my personal macbook seems to have no recognized me as well me... lemme see if I can figure that out

Turns out my macbook has git set up with Ashley Smith<EMAIL_ADDRESS>as my username instead of ashley3 so I'm gonna try to kill this pull request and undo that commit


A11ybugs: Fixed most of the Accesibility bugs on AMP Start

Everything was fixed using Chromevox, I am closing some of the Mobile specific issue, since often times, they're were cause by the same problem.
closes #373
closes #388
closes #418
closes #409
closes #402
closes #383
closes #414
closes #413
closes #411
closes #417
closes #419
closes #395
More may have been closed, as many are duplicated on different devices. If there are, please let me know as I am tracking this personally.

I rebased the PR to not include my @google git config, my apologies, didn't notice beforehand

Commenting to get cla/google to run


Include VK_hdr_layer?

While it's not exactly part of KDE it would be interesting to have VK_hdr_layer (temporarily) packaged alongside plasma6, since AFAIK kwin implements the relevant (temporary/experimental) wayland protocols.
AIUI this layer + kwin6 should allow mpv and other capable vulkan apps to output HDR frames.
(this is mostly for my personal interest in playing around with it ahead of full support in kwin, so no problem if you're not interested in dragging the layer along with the KDE stuff)

Hmm, I guess I misinterpreted some of the commit messages over at the kwin repo. Gamescope running as a wayland client even explicitly prints that the compositor does not support hdr. Guess I'll just look into upstreaming the gamescope update and put this on the backburner again.

Ah, got it to work! It just needs this fork rather than the original, since the original implements the WIP color management wayland protocol and the fork implements the unofficial "frog color management protocol" that the kwin_wayland beta supports for testing.

Hey @jansol mind sharing how you got this to work? I'm also interested in testing this as well

I just added a package for it it to my local nixpkgs, basing the derivation on the existing one from vulkan-extension-layer. Then I added that package to my systemPackages and that's it.

Thanks for the tip @jansol
I've tried this:
{ lib, stdenv, fetchFromGitHub, cmake, pkg-config, writeText, vulkan-headers, vulkan-utility-libraries,  jq, libX11, libXrandr, libxcb, wayland }:

stdenv.mkDerivation rec {
  pname = "vulkan-hdr-layer";
  version = "63d2eec";

  src = fetchFromGitHub {
    owner = "Drakulix";
    repo = "VK_hdr_layer";
    rev = "63d2eeccb962824c90e158a06900ae1abec9c49e";
    fetchSubmodules = true;
    hash = "sha256-rSKPTeTDOz6IeJGRt9aIu1VH8VfVzXNYZfjdiSXEJxg=";
  };

  nativeBuildInputs = [ cmake pkg-config jq ];

  buildInputs = [ vulkan-headers vulkan-utility-libraries libX11 libXrandr libxcb wayland ];

  # Help vulkan-loader find the validation layers
  setupHook = writeText "setup-hook" ''
    addToSearchPath XDG_DATA_DIRS @out@/share
  '';

  # Tests are not for gpu-less and headless environments
  cmakeFlags = [
    "-DBUILD_TESTS=false"
  ];

  # Include absolute paths to layer libraries in their associated
  # layer definition json files.
  preFixup = ''
    for f in "$out"/share/vulkan/explicit_layer.d/*.json "$out"/share/vulkan/implicit_layer.d/*.json; do
      jq <"$f" >tmp.json ".layer.library_path = \"$out/lib/\" + .layer.library_path"
      mv tmp.json "$f"
    done
  '';

  meta = with lib; {
    description = "Layers providing Vulkan HDR";
    homepage = "https://github.com/Drakulix/VK_hdr_layer";
    platforms = platforms.linux;
    license = licenses.mit;
  };
}

But I'm not seeing the JSON ls /run/current-system/sw/share/vulkan/explicit_layer.d/ which is making me suspect I did something wrong. I'm trying to run moonlight with HDR

It turns out I was caching vulkan-extension-layer because I forgot to update the hash.. Oops :) this actually show's what's expected.. Let's see if it works.
{ lib, stdenv, fetchFromGitHub, meson, pkg-config, vulkan-loader, ninja, writeText, vulkan-headers, vulkan-utility-libraries,  jq, libX11, libXrandr, libxcb, wayland }:

stdenv.mkDerivation rec {
  pname = "vulkan-hdr-layer";
  version = "63d2eec";

  src = (fetchFromGitHub {
    owner = "Drakulix";
    repo = "VK_hdr_layer";
    rev = "63d2eeccb962824c90e158a06900ae1abec9c49e";
    fetchSubmodules = true;
    hash = "sha256-IwHrMTiOzITMsGMZN/AuUN3PF/oMhENw9d7kX2VnDGM=";
  }).overrideAttrs (_: {
    GIT_CONFIG_COUNT = 1;
    GIT_CONFIG_KEY_0 = "url.https://github.com/.insteadOf";
    GIT_CONFIG_VALUE_0 = "git@github.com:";
  });

  nativeBuildInputs = [ vulkan-headers meson ninja pkg-config jq ];

  buildInputs = [ vulkan-headers vulkan-loader vulkan-utility-libraries libX11 libXrandr libxcb wayland ];

  # Help vulkan-loader find the validation layers
  setupHook = writeText "setup-hook" ''
    addToSearchPath XDG_DATA_DIRS @out@/share
  '';

  # Include absolute paths to layer libraries in their associated
  # layer definition json files.
  preFixup = ''
    for f in "$out"/share/vulkan/explicit_layer.d/*.json "$out"/share/vulkan/implicit_layer.d/*.json; do
      jq <"$f" >tmp.json ".layer.library_path = \"$out/lib/\" + .layer.library_path"
      mv tmp.json "$f"
    done
  '';

  meta = with lib; {
    description = "Layers providing Vulkan HDR";
    homepage = "https://github.com/Drakulix/VK_hdr_layer";
    platforms = platforms.linux;
    license = licenses.mit;
  };
}

From what I recall you need to use Zamundaaa's branch here. This is the script that eventually got me up and running for HDR in Quake II RTX.
For my own curiosity I also tested on Cyberpunk 2077 and EA Sports WRC to no luck, as expected.
{ lib, stdenv, fetchFromGitHub, meson, pkg-config, vulkan-loader, ninja, writeText, vulkan-headers, vulkan-utility-libraries,  jq, libX11, libXrandr, libxcb, wayland }:

stdenv.mkDerivation rec {
  pname = "vulkan-hdr-layer";
  version = "63d2eec";

  src = (fetchFromGitHub {
    owner = "Zamundaaa";
    repo = "VK_hdr_layer";
    rev = "869199cd2746e7f69cf19955153080842b6dacfc";
    fetchSubmodules = true;
    hash = "sha256-xfVYI+Aajmnf3BTaY2Ysg5fyDO6SwDFGyU0L+F+E3is=";
  }).overrideAttrs (_: {
    GIT_CONFIG_COUNT = 1;
    GIT_CONFIG_KEY_0 = "url.https://github.com/.insteadOf";
    GIT_CONFIG_VALUE_0 = "git@github.com:";
  });

  nativeBuildInputs = [ vulkan-headers meson ninja pkg-config jq ];

  buildInputs = [ vulkan-headers vulkan-loader vulkan-utility-libraries libX11 libXrandr libxcb wayland ];

  # Help vulkan-loader find the validation layers
  setupHook = writeText "setup-hook" ''
    addToSearchPath XDG_DATA_DIRS @out@/share
  '';

  meta = with lib; {
    description = "Layers providing Vulkan HDR";
    homepage = "https://github.com/Zamundaaa/VK_hdr_layer";
    platforms = platforms.linux;
    license = licenses.mit;
  };
}

Awesome to hear! I'm also having bad-luck with Wayland. I tried mpv yet.

Switched from using a modified nixpkgs to just defining the package inline in my home-manager user packages list. Makes it much easier to track upstream nixos-unstable.

Good idea, I also have it imported in an external module. Makes it easier when this and KDE 6 eventually get releases and the cleanup needs to happen.


Mysql flexible-server restore failed

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az mysql flexible-server restore
Errors:
(OperationCancelled) The operation has been cancelled by user.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az mysql flexible-server restore --debug -g {} -n {} --source-server {} --restore-time {}

Expected Behavior
Environment Summary
Linux-5.4.0-1048-azure-x86_64-with-debian-10.2 (Cloud Shell)
Python 3.6.10
Installer: DEB

azure-cli 2.24.2

Extensions:
ai-examples 0.2.5
ssh 0.1.5

Additional Context
The command runs  for 16min and then stops with the error-message above...
No action/input is taken from the user --> don't understand the error-message!
Timeout ?
The debug-logs az.log and telemetry.log are available.

route to service team

Thanks for reporting this. Adding @dimadhus to help with this.

Ack!

Hello,
has there been any progress in solving this problem?

@yonzhan Do we have additional information like operation id / client request id to investigate the issue.
The CLI command seems to work fine, the documentation on the Az cli command is correct.
We want to understand how many restore attempts did the customer make to root cause.

Hunch is an operation with the server name is already in progress but stuck and we may be throwing incorrect error message for a specific scenario.


API documentation

Have you considered refactoring your api code into a standalone library?

Obviously not trying to gravedig, but I've been looking for people on my issue SuperSpyTX/Hack-Yak/issues/6 - I know soren121 will volunteer when he has time.