ReaderThreadError: wrong number of arguments (given 0, expected 1)

Your environment

ruby -v: 3.3.6
rdbg -v: 1.9.2
Inside docker (docker image ruby:3.3.6-slim-bookworm@sha256:7a89be75c6768a888f70c90f4626366de90909c4e4d62729775ea3b64672547c to be exact)
Process started like: bundle exec rdbg -O -n --session-name='web' -c -- bin/rails server -b <IP_ADDRESS> -p 3000

RUBY_DEBUG_SOCK_DIR=/app/tmp (Rails root being /app inside container, /app/tmp is a volume mounted into container so socket can be attached to from outside container)

Describe the bug
Quite often during the course of debugging the debugger process seems to becomes in a paused state but unable to display the current thread or breakpoint. Rails requests hang even if disconnecting from the remote debugger (VSCode connecting via a socket).
After that point the main process becomes unresponsive and the only way to continue is to restart the parent process (which in our case means restarting the docker container).
To Reproduce
We have seen this arise sporadically. The following is one way we've been able to reproduce it but we have seen it occur even when avoiding the below:

When paused at a breakpoint, calling another method manually (or in a watch) which has its own breakpoints set will cause execution to run to and pause at the nested breakpoint, but continuing from there will break the debugger.
class Foo
  def a
    puts "hello"    # BP 1 <= active
    1 + 1
  end

  def b
    42              # BP 2
  end
end
Foo.new.a

Given the following file with breakpoints set above, when stopped at BP 1, calling b will re-break at BP 2, but continuing from there seems to break the debugger and leave some thread in a hung state.

Again, this is just one cause but we have been able to reproduce it at times without any manual execution of methods with BPs. However, we do not have a reproduction case for this.
Expected behavior
The debugger doesn't hang
Additional context
When this is encountered, I disconnected VSCode's debugger and manually attached to the same debugger instance via rdbg -A socket. When doing so the debugger output shows:
root@b032c5a1c83b:/app# rdbg -A tmp/rdbg-7-web
(unknown) "Content-Length: 154\r\n"
(unknown) "\r\n"
(unknown) "{\"type\":\"event\",\"event\":\"output\",\"body\":{\"category\":\"console\",\"output\":\"DEBUGGER (client): Connected. PID:7, $0:bin/rails, session_name:web\\n\"},\"seq\":434}"
root@b032c5a1c83b:/app# # ^ exits

The application logs show:
DEBUGGER: Connected.
DEBUGGER: ReaderThreadError: wrong number of arguments (given 0, expected 1)
["/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server_dap.rb:504:in `puts'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:156:in `greeting'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:56:in `block (3 levels) in activate'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:54:in `synchronize'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:54:in `block (2 levels) in activate'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:506:in `block in accept'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:815:in `block (2 levels) in accept_loop'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:812:in `each'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:812:in `block in accept_loop'",
 "<internal:kernel>:187:in `loop'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:810:in `accept_loop'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:1170:in `block in unix_server_loop'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:1124:in `unix_server_socket'",
 "/usr/local/lib/ruby/3.3.0/socket.rb:1169:in `unix_server_loop'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:502:in `accept'",
 "/usr/local/lib/ruby/gems/3.3.0/gems/debug-1.9.2/lib/debug/server.rb:49:in `block in activate'"]
DEBUGGER: Disconnected.

ReaderThreadError: wrong number of arguments (given 0, expected 1) will be solved with https://github.com/ruby/debug/pull/1124, but the original issue (hanging randomly) is not fixed.
Sorry for inconvenient. I also got similar issue but no idea how to debug it...

Ah the other piece of context here is that the web app is using Puma. I saw https://github.com/ruby/debug/issues/1027#issuecomment-1986761959 mentions some rough interactions there, so we I am experimenting with setting min_threads, max_threads = 1, 1 in Puma to see if it decreases the likelihood of these kinds of hangs.


Terminal cursor is bouncy in codespaces

Using a codespace the terminal cursor appears to bounce around in position while typing.

does changing terminal.integrated.localEchoLatencyThreshold help?

/duplicate https://github.com/microsoft/vscode/issues/126209


Telescope not focused when vim.ui.select is triggered by autocmd

Describe the bug
When setting up an autocmd to automatically format the buffer on save with the LSP, if there are multiple formatting capable language servers vim.ui.select is called to select the correct one. The Telescope selection window appears but it not in  focus
System information

OS: linux
Neovim version:

NVIM v0.7.2
Build type: Release
LuaJIT 2.1.0-beta3

Is this related to a specific vim.ui.select backend? If so, which one? Yes, telescope
Dressing config:

require("dressing").setup({
  input = {
    enabled = ui_opts.nui_input,
    default_prompt = "➤ ",
    winhighlight = "Normal:Normal,NormalNC:Normal",
  },
  select = {
    enabled = ui_opts.telescope_select,
    backend = { "telescope", "builtin" },
    builtin = { winhighlight = "Normal:Normal,NormalNC:Normal" },
  },
})

To Reproduce
Steps to reproduce the behavior:

Set up dressing
Add an  autocmd to format synchronously on save (BufWritePre)

on_attach = function(client, bufnr)
  if client.resolved_capabilities.document_formatting then
    vim.api.nvim_create_autocmd("BufWritePre", {
      desc = "Auto format before save",
      pattern = "<buffer>",
      callback = function() vim.lsp.buf.formatting_sync() end,
    })
  end
end,

Have multiple language servers with formatting capabilities. For example sumneko_lua and stylua (with null-ls)
Open a file using at least 2 format capable language servers and save the file.

Screenshots
https://asciinema.org/a/86k1OuM0mj4hi8fQLTt7qz6B5

I think this is because of a recent change I made for #58. I've reverted the vim.ui.select part of it. Can you give it another try?

That worked perfectly! Thank you so much!


requirejs undefined is not a function

I have snap.svg working with requirejs fine until I optimise when I then get the error undefined is not a function in relation to
return eve("snap.util.getattr." + params, el).firstDefined();

any suggestions as to how I can fix this?

I'm facing same issue, any progress?


Only Julia1.7 permitted

It's a really nice pkg, but would you plz add more versions to make it more compatible.
I think that a lot of users are now using julia1.0~julia1.6 versions. Thanks :)

Hey there!
Thanks for the suggestions. We can certainly try to provide some features for lower Julia versions. However, for the peakflops tools we rely on a recent version of LLVM etc. and therefore a modern Julia version. (Actually, to test Float64 WMMAs on Tensor Cores Julia 1.7 isn't even enough but we need Julia 1.8 or later.)
Also note that we likely won't support Julia < 1.6 because 1.6 is the current LTS and people using older Julia versions should seriously consider upgrading to 1.6.)
Best,
Carsten

We provide initial 1.6 support (as far as possible) from version 0.2.0 on (will release it later today).


Not friendly with webpack

I try to use taiko as a feature of an electron app which is created with electron-forge ts webpack template.
After bundling, it crashed. First, the package.main refer to a bin file which contains # symbol, which is not a valid js symbol.
After fixing this by import from taiko/lib/taiko, it crashed again, since it requires package.json to get the chromium revision, but after bundling, it cannot find package.json.
I stopped here and switched to use puppeteer-core instead, since taiko seems not been built as a general library, but has some assumption about the runnning environment.
I like taiko's easy api than puppeteer, if they will work in my case, I'd glad to use it to replace puppeteer. Thanks for you guys' great job~
Versions
Taiko Version: 1.0.3 (Chromium: 81.0.3994.0) RELEASE
Node: v12.14.0

Do you have an example project where importing taiko as a lib fails? We'd like taiko to behave with module loaders, so if there's anything to improve it's good to know. thanks.

Currently no, but I can build one in my spare time.
Another design to make it not good as a lib is the auto-chromium-downloading. Refer to puppeteer-core, it would not do this automatically but instead exposes an api to enable you if you wish.

Makes sense. I guess that would involve breaking taiko into sub-package, so that the repl can continue to download chromium, whereas the core can expose the functionality.

@sriv
https://github.com/zhaoyao91/demo-electron-taiko-bug

@zhaoyao91 #1408 fixes this issue to raise the right error.
You can try the change along with changing
https://github.com/zhaoyao91/demo-electron-taiko-bug/blob/da0d9312facaf3a558afd2c5167cf220d8354045/src/try-taiko.ts#L7
To
process.env.TAIKO_BROWSER_PATH = chromePaths.chrome;

Another design to make it not good as a lib is the auto-chromium-downloading. Refer to puppeteer-core, it would not do this automatically but instead exposes an api to enable you if you wish.

You can skip downloading chromium by setting the environment variable
TAIKO_SKIP_CHROMIUM_DOWNLOAD=true


CPP: Clean up and fix FormattingFunction, FormatLiteral

This PR cleans up some of the FormattingFunction and FormatLiteral logic, in particular removing the old isWideCharDefault() mechanism which has been superseded by the getDefaultCharType() mechanism (but we never got around to removing the former).  With just one mechanism, it should be easier to go on to fix the (platform dependent) handling of %S (and %C) that's been discussed recently.
Also fixed a bug I found in FormattingFunction.getDefaultCharType().

LGTM. I'm happy to merge this when the internal PR is ready.


Results of discussion regarding GitHub procedures

This issue is to record the discussion that ensued around GitHub use procedures at the 2020 CF Community Meeting on 2020-06-09. I have copied it from the plenary discussion notes and will use it to generate issues that can be closed by setting things up in GitHub or noting good practices in CONTRIBUTING.md. Apologies for the noise - normally we like 1 issue for 1 topic but I prefer to approach this in this way in order to ensure that no contributions are lost.

[x] Sadie Bartholomew (NCAS/Reading): What would, or have, you done in the case that an Issue gets so many comments it becomes too impractical to follow?
[x] [SB as above]: Is the 3 week rule rigid (& fairly precisely held)? I am wondering about times such as over Christmas etc when people might be less active such that it could be ‘easier’ to get acceptance under that part of the rule.

[x] Hélène Côté (Ouranos): I think flexibility is nice as the need to reach a diversity of point of views and other communities will increase. At least the date of the end of this period should be explicit.
[x] SB: I would agree (at least some) flexibility is a plus.
[x] Jonathan: I think it’s OK, within the three-week period, to “object” by saying, “I need more time.” The most important thing is not the total time it took, but that the best possible consensus is achieved.
[x] Hélène Côté(Ouranos): I agree.

[x] David Hassell (NCAS): Who would be responsible for moving the issue between columns of the project board? The proposer, the moderator, or someone else? (I like the idea, by the way.)
[x] Trevor Smith (Ouranos): Have you considered using the automated features for moving KanBan cards? IE: Setting a rule that once a PR is accepted, it can move to a new column. I’m not sure if you can make time-based rules, but that could help reduce the human effort needed.

[x] (Trevor Smith) I also like the idea of asking contributors to add their names to History and Contributors files. We currently do this for some projects.
[x] David Hassell: I too like the idea of asking the proposer to do this.

[x] Karl Taylor (PCMDI):  I liked the suggested technical changes for improving the process.
[x] Trevor Smith (Ouranos): Minor suggestion, but if you use the html commenting conventions ( <-- Comments go here -->) contributors won’t need to remove them when adding a new issue/PR.
[x] Jonathan Gregory, Daniel Lee: note moderator in history? Keep summary up to date but also post updates as comments? 6 week wait triggered by motion?

[x] David Hassell: Also note proposer in history, so contribution is made specific.

[x] Kanban seems to have support as voiced by Trevor & Klaus - introducing it does not cause harm
[x] Jonathan: To record David’s suggestion of making a comment in the issue when the summary is updated, in order that everyone will be notified, and also because the moderator’s summary is part of the discussion and needs to be recorded therein for the discussion to make sense when read subsequently

To record David’s suggestion of making a comment in the issue when the summary is updated, in order that everyone will be notified, and also because the moderator’s summary is part of the discussion and needs to be recorded therein for the discussion to make sense when read subsequently

Hi @JonathanGregory , good points - I think this would best be served in CONTRIBUTING.md as it's an implementation item, not necessarily a rule. I've spawned #362 to follow this. I'm closing this issue in favour of that one, noting that they're now linked.


Scripts are out of date and have proliferated

We should re-prioritize and decide what stays (a little) and goes (a lot). Might be worth providing a script-ified version of the functionality from the tutorial.

Scripts have been trimmed in number. Remaining need updating to current "time" syntax

Addressed in PR #25


[improvement](jdbc catalog) Delete unnecessary schema and optimize insert logic

Proposed changes
Issue Number: close #xxx
In the previous design, we were compatible with MySQL's auto-increment column and default value to bypass the null value check when writing back Jdbc External Table. However, because MySQL's default value is not completely unified with Doris, this resulted in The unsuitable default value is wrong. In response to this situation, I made the following optimizations

For JDBC External Table, we always allow certain columns to be missing during insertion. Even if these columns are not allowed to be empty at the source end, the error should be generated by the source end, not Doris herself.
When the target column is non-nullable and the insertion is done via INSERT INTO tbl VALUES() or INSERT INTO tbl SELECT constants, Doris should verify any inconsistency between them and throw an exception. This check is not applied for INSERT INTO tbl SELECT ... FROM tbl operations.

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall


CodeShip doesn't seem to run the Smoke Tests

Describe the bug
RealSmokeTest has more than 20 tests yet Codship build reports
Running org.zendesk.client.v2.RealSmokeTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.002 sec - in org.zendesk.client.v2.RealSmokeTest

To Reproduce
Look at any of the builds in https://app.codeship.com/projects/302087
Expected behavior
Smoke Tests are executed.

I reconfigured it but not sure if it will work I cannot launch CodeShip builds on PRs :-(

https://app.codeship.com/projects/302087/builds/42167376?pipeline=2cab6195-3724-4a83-90fc-0f0434a47870
It's now ok (and I have the same error locally)


Angular valuechanges

I have 5 form controls inside the formgroup and I have subscribed the valueChanges method on formGroup.
How can I opt out one formControl from that formGroup not to subscribe with valueChanges? because I have separated logic for that control and that logic will take care all the required things.
Example:
this.myForm = this.fb.group({
id: [, { updateOn: 'change' }],
name: [, { updateOn: 'change' }]
})
this.myForm.valueChanges.subscribe(x=> this.saveChages());
But let's say from the above subscription what if I want to opt out name?
FYI: I would like to avoid subscription for each and every control separately. It led to increase the lines of code so.
Thanks in advance.
Dhimant

Another example
ngOnInit(){
this.myForm = this.fb.group({
	id: [, { updateOn: 'change' }],
    name: [, { updateOn: 'change' }],
    city: [, { updateOn: 'change' }]
})

this.myForm.valueChanges.subscribe(x=> this.saveChages());

this.myForm.get('city').valueChanges.subscribe(x=> this.saveChagesCity());

}
saveChages(): void {
//saving the changes
}
saveChagesCity(): void {
//saving the changes business logic only for city
}
QUESTION:
Here I want to opt out the "city" formControl from calling the saveChages() method.
In the above code, as soon as I change the city, both method is being called which I don't want it.

This has nothing to do with this repository or module.


Provide typings for modules

The Problem
Currently modules can be instantiated like so:
// Passing an empty object to the annotation
// can become a hassle. Maybe we should create an overload that accepts 0 args, or just
// a default value equal to {}
@Module({})
class MyModule {
  onModuleInit () {
    // more stuff here
  }
}

Possible solutions
Is there any way to extend the type on the annotation, so onModuleInit would autocomplete?
Maybe we could provide an interface for the typings too, but it would require the user to import them:
import { Module, IModule } from 'injets'
@Module({})
class MyModule implements IModule {
}

It looks like C# though.

Injets exports an OnModuleInit interface, just implement it.
import { Module, OnModuleInit } from 'injets'

@Module({})
class MyModule implements OnModuleInit {
  onModuleInit () {
    // your code here
  }
}


CalloutClickedCommand

Hello,
I don't understand, this command to active when I go load de maps...
does not have to be activated when I click on the information window of the selected pin?
I'm confused, any solution?

I'm sorry, this mistake was mine! all ready.

I don't understand this command. it is supposed to get triggered when you click the pin correct? I would like to open a different view when they click on the Callout command. any ideas?


support permenant, destructive method to remove the table

Can support permenant, destructive method to remove the table ?

Yes, a general method is to extract the firmware with a programmer, remove the table or embed this program as a driver into the firmware (whatever is feasible) and reprogram the SPI flash. Note:

exact steps vary from board to board, so I cannot give you any further help regarding the specific board you are using
this is dangerous and you might damage the motherboard on the way
If PSB, Sure Start or other similar technology is enabled, you might not be able to boot a modified firmware at all


minify removes "%" from inline style in HTML

So, I'm not sure whether this is intended behaviour, but I stumbled upon an issue recently caused by a missing "%" in an inline style.
After looking for a while, I found this is caused by minify. I made a simple way to reproduce.
Caddyfile:
localhost:8080 {
	root site
	minify
}

HTML page:
<!doctype html>

    <title>minify-test</title>

when using the minify directive, left: 0% becomes left: 0
I realise that in this case left: 0 and left: 0% are identical, but in the production case, the missing "%" was breaking a script.

Sorry, but this issue is not directly related to this repository. You should move this issue to tdewolff/minify which is the engine we use 😄 Thanks!


The same go module + js code works on node.js but not electron.
This is how I'm building the go module, (someone has a nice example of this working with node.js here: https://github.com/MrSaints/go-node-ffi):
go build -buildmode=c-shared -o go-node-ffi.dylib
Simple, basic electron app:
var app = require('app');  // Module to control application life.
var ffi = require('ffi');
[..]
app.on('ready', function() {
  // Create the browser window.
  mainWindow = new BrowserWindow({width: 800, height: 600});
  [...]
  var go = ffi.Library("gomodule", {
      "Add": ["int", ["int", "int"]]
      });

The exception is:
runtime/cgo: could not obtain pthread_keys
	tried 0x118 0x119 0x11a 0x11b 0x11c 0x11d 0x11e 0x11f 0x120 0x121 0x122 0x123 0x124 0x125 0x126 0x127 0x128 0x129 0x12a 0x12b 0x12c 0x12d 0x12e 0x12f 0x130 0x131 0x132 0x133 0x134 0x135 0x136 0x137 0x138 [...]

Sounds like it should be possible to get this working on Electron, if someone can point me to the right direction, I can try and fix this and put up a pull request!
I'm on MacOS.
Thanks,

Chromium itself is a much more complicated environment than Node.js, from what I can find go-lang runtime is using some hacky way to initialize the environment, which seems to conflict with Chromium's environment.
So I suggest using the module in scripts forked by child_process.fork, which is very similar to upstream Node's environment. The root cause is in go-lang's internals, so unless some core go-lang developer looks into this we are not able to solve it.

Using child_process.fork as @zcbenz said worked for me. Thanks a lot!!!


After checking this, you need to ./hack/update-deps.sh --upgrade && ./hack/update-codegen.sh and then check for diffs.

After checking this, you need to ./hack/update-deps.sh --upgrade && ./hack/update-codegen.sh and then check for diffs.
Originally posted by @mattmoor in https://github.com/knative-sandbox/discovery/pull/80#discussion_r498563624

/close

/close


TestSimple/linux/clean/busybox: failing due to overridden "none" results

It looks like our tests are adding none results to the simple output for overridden rules, unlike make refresh-sample-testdata:
    --- FAIL: TestSimple/linux/clean/busybox (0.08s)
        samples_test.go:160: Simple output mismatch: (-want +got):
              (
              	"""
              	... // 63 identical lines
              	fs/watch: low
              	impact/ddos/raw_flooder: medium
            + 	impact/degrade/infection: none
            + 	impact/degrade/linux_paths: none
            + 	impact/remote_access/reverse_shell: none
              	lateral/scan/tool: medium
              	net/ip/addr: medium
              	... // 24 identical lines
              	"""
              )

Example: https://github.com/chainguard-dev/malcontent/actions/runs/11676935450/job/32514021648?pr=584#step:8:420
Blocks #584

If I scan busybox locally I don't see any behaviors with none for their severity:
go run cmd/mal/mal.go --format simple analyze ./out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534/linux/clean/busybox
# out/samples-ec1ba5f2dc0e1f7085a0af73aa0f6fb1043e7534/linux/clean/busybox: medium
c2/addr/ip_port: medium
credential/os/gshadow: medium
credential/os/shadow: medium
credential/password: low
data/compression/bzip2: low
data/compression/gzip: low
data/encoding/base64: low
data/random/insecure: low
discover/group/lookup: medium
discover/network/interface_get: low
discover/network/netstat: medium
discover/processes/pgrep: medium
discover/system/cpu_info: low
discover/system/platform: low
discover/system/sysinfo: medium
discover/user/HOME: low
discover/user/USER: low
discover/user/name_get: medium
evasion/hide_artifacts/pivot_root: medium
evasion/hijack_execution/LD_LIBRARY_PATH: low
evasion/logging/current_logins: medium
evasion/process_injection/ptrace: medium
exec/conditional/LANG: low
exec/program: medium
exec/program/background: low
exec/reconfigure/hostname_set: low
exec/shell/SHELL: low
exec/shell/TERM: low
exec/shell/exec: medium
exec/tty/parameters_get: medium
exec/tty/pathname: medium
false-positives/busybox: low
fs/directory/create: low
fs/directory/remove: low
fs/fifo_create: low
fs/file/copy: medium
fs/file/delete: low
fs/file/times_set: medium
fs/file/truncate: low
fs/link_read: low
fs/lock_update: low
fs/mount: low
fs/mounts_read: medium
fs/node_create: low
fs/path/etc: low
fs/path/home: medium
fs/path/tmp: medium
fs/path/usr_bin: low
fs/path/usr_sbin: low
fs/path/var: low
fs/path/var_log: medium
fs/permission/chown: low
fs/permission/modify: medium
fs/proc/arbitrary_pid: medium
fs/proc/meminfo: medium
fs/proc/mounts: medium
fs/proc/stat: medium
fs/symlink_resolve: low
fs/tempdir: low
fs/tempdir/TMPDIR: low
fs/tempdir/create: low
fs/tempdir/tempfile_create: low
fs/watch: low
impact/ddos/raw_flooder: medium
lateral/scan/tool: medium
net/ip/addr: medium
net/ip/icmp: medium
net/ip/multicast_send: low
net/ip/parse: medium
net/ip/string: medium
net/ip/tcp_state_tracker: medium
net/resolve/hostname: low
net/resolve/hostport_parse: low
net/socket/local_addr: low
net/socket/raw_s: medium
net/socket/receive: low
net/socket/send: low
net/url/embedded: low
os/time/clock_set: low
persist/daemon: medium
process/chdir: low
process/chroot: low
process/create: low
process/executable_path: low
process/groupid_set: low
process/groups_set: low
process/namespace_set: low
process/parent_pid_get: low
process/userid_set: low

Scanning busybox from your branch also doesn't show those findings. 🧐

Ah, okay. We're setting the override severities to ignore. I'll fix this up.
To catch this (and the offending rules) I had to run:
$ go run cmd/mal/mal.go --format strings --min-risk any --ignore-tags harmless analyze ./out/samples-e58368a24b930f7dcf555678a8bc63f9d45aef24/linux/clean/busybox


Easier way to draw a circular segment in d3?

I found d3 has d3.svg.arc for convenient drawing of an arc.
However, there seems no way to draw a circular segment ⌓ easily.
The possible parameters for drawing circular segment are radius (r) and height (h).
Will it be possible for this shape to be added to d3 in the future?

No immediate plans to do this, but it would go into d3-shape or a new D3 plugin if it were to be added.
Also see my Circular Segment example, which uses clipping. But I think it’d be pretty straightforward to change it to use SVG’s elliptical arc commands.

Hey!  I wrote a plugin which could help you: https://github.com/chandramouli-sastry/d3-csegment. I'm quite new to open source and I'm working on creating a pull request. Please let me know what you think about this.


Github Actions Added

Fixes #[issue number that will be closed through this PR] 🔐
Are you a part of LGM SOC' 21 ❓

[x] Yes ✔
[ ] No ✖

Describe the changes you've made 🏆
A clear and concise description of what you have done to successfully close your assigned issue. Have you added any new files? Any changes that you think are relevant to be added. Feel free to let us know! 😊

Stale Bot
Husky to run ESLint at pre-commits and pre-pull-requests
DeepSource
Welcome and Greetings Bot

Type of change 🖇
Please delete options that are not relevant.

[ ] Bug fixing (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Code style update (formatting, local variables)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Checklist: 📋

[x] My code follows the style guidelines of this project.
[x] I have performed a self-review of my own code.
[ ] I have commented my code, particularly in hard-to-understand areas.
[ ] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.
[ ] Any dependent changes have been merged and published in downstream modules.

Screenshots (if required) 📷
Add screenshots of the changes you have made.

Original
Updated

original screenshot
updated screenshot

@kanak22 @yashsehgal why this PR is not reviewd till now?

There is some work going on on a new version of the project that is why it is not yet reviewed and merged

Hey @RituCs 👋🏽 We are happy that you decided to contribute to this project of ours. Actually, all the projects under @DesignSystemsOSS are temporarily archived and are under planning/re-structuring. That's why we aren't not accepting any changes. Hope you'll understand :D


Upgrade Python scripts to Python 3

Python 2 is pretty obsolete nowadays, and now that we dropped Construct 2 as a dependency it shouldn't be hard to port. We should recommend pillow instead of PIL in the README.

We might just support /both/ Python 2 & 3. That way we can use PyPy2 or CPython 3.


[SPARK-30594][CORE] Do not post SparkListenerBlockUpdated when updateBlockInfo returns false

What changes were proposed in this pull request?

If updateBlockInfo returns false, which means the BlockManager will re-register and report all blocks later. So, we may report two times for the same block, which causes AppStatusListener to count used memory for two times, too. As a result, the used memory can exceed the total memory.
So, this PR changes it to not post SparkListenerBlockUpdated when updateBlockInfo returns false. And, always clean up used memory whenever AppStatusListener receives SparkListenerBlockManagerAdded.
Why are the changes needed?

This PR tries to fix negative memory usage in UI (https://user-images.githubusercontent.com/3488126/72131225-95e37e00-33b6-11ea-8708-6e5ed328d1ca.png, see #27144 ). Though, I'm not very sure this is the root cause for #27144 since known information is limited here.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Honestly, I have no good idea to write a unit test for this issue yet.

@xuanyuanking Can you cherry-pick your commits to this branch? They actually looks good.

During the cherry-pick, please don't forget to fix the invalid UT, @xuanyuanking .

@dongjoon-hyun The UT has been fixed by https://github.com/apache/spark/pull/27418/commits/fecaf20a4b8c88596f6fd60e5033b9b4e328b623

Oh. I missed that. Thanks, @jiangxb1987 ! If that is merged here, I'll test once more.

@jiangxb1987 Thanks Xingbo, the cherry-picks were done in https://github.com/Ngone51/spark/pull/1, waiting for @Ngone51 merging.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117759/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22521/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117760/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22522/
Test PASSed.

Jenkins, retest this please.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117761/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22523/
Test PASSed.

@dongjoon-hyun Would you please help check Jenkins failure?

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22525/
Test PASSed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117765/
Test FAILed.

retest this please

Test build #117773 has started for PR 27306 at commit 5f12317.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/22535/
Test PASSed.

Test build #117773 has finished for PR 27306 at commit 5f12317.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/117773/
Test PASSed.

thanks, merging to master/3.0!

thanks all, especially for @xuanyuanking !


Simplify minos.saga.SagaStepStatus

Closing this issue as it is moved to https://github.com/minos-framework/minos-python/issues/53


Shared element transition in SummaryProductFragment

Description
Added the shared element transition in SummaryProductFragment for products.
Related issues and discussion
#1277
Screen-shots, if any

Checklist
Make sure you've done all the following (You can delete the checklist before submitting)

[x] Apply the AndroidStyle.xml style template to your code in Android Studio.
[x] Code is well documented
[x] Include unit tests for new functionality
[x] All user-visible strings are made translatable
[x] Code passes Travis builds in your branch
[x] If you have multiple commits please combine them into one commit by squashing them.
[x] Read and understood the contribution guidelines .

@teolemon @Karljoones Please review :)

Hey,
@Karljoones @gulzar1996 Thanks for your review. I have updated my PR accordingly. Please review.
Thanks :+1:

Merged, thanks @amanjeetsingh150


Old Credentials Deprecated Without Warning, Nowhere To Get New Ones

Per the title, the old API was shut off without any sort of notice. Following the implementation guide for the new API, and passing the old credentials, does not work. There is nowhere to apply for new credentials.
We have emailed<EMAIL_ADDRESS>and David Kaminsky<EMAIL_ADDRESS>with no replies.
Please advise.

+1

Had a similar experience. Previously obtained token was throwing deprecated error so I had to try the new guideline for getting new access token but had no luck, always getting 400 see specific error below. Imo, the API docs is kinda lacking and needs improvement.
{
    "error": "invalid_client",
    "error_description": "Client credentials are invalid"
}

The docs seem like a direct copy of Viagogo's docs: https://github.com/viagogo/viagogo-api-docs which have the same problem ("nowhere to register").
Presumably this is all related to Viagogo and Stubhub merging not long ago.

@chriskallen @taddison @wayneduke @JamesHoldsworth

@Greeeeyyyss finally heard back from API.Support, they are "not opening up any of the data APIs". So this is now a dead end for us.

@SpikedCola appreciated the heads up! Unfortunately they hadn't responded yet in my email<EMAIL_ADDRESS>


Please update pod and carthage to support swift 5

Getting error after i have updated my xcode to 10.2 ,swift 5
Error : Module compiled with Swift 4.2.1 cannot be imported by the Swift 5.0 compiler: /Projects/Discovrus/APP/Discovrus/Carthage/Build/iOS/Parchment.framework/Modules/Parchment.swiftmodule/arm64.swiftmodule

That means that you just need to rebuild the framework with Xcode 10.2: carthage build --platform ios

Issue has been resolved.
Thanks @ikesyo 👍

sorry, this is still an issue for us. I have uninstalled and reinstalled both Xcode and Carthage and I am still getting:
error: SWIFT_VERSION '3.0' is unsupported, supported versions are: 4.0, 4.2, 5.0. (in target 'MobileFreddy')
when I run Carthage build --platform iOS


AlphaFold: Usage for single sequence?

As AlphaFold is intended to predict structure from a sequence, I want to input [some sequence of letters denoting an arbitrary amino acid sequence], and receive [a PDB file of the predicted structure(s)] as output. Can some part of the released code do this, using the released precomputed features? It is unclear from the readme how to use this code beyond replicating the CASP13 results.
For example, this modelling server does exactly that, but by method of comparing sequence similarity to existing homology models:
https://gomodo.grs.kfa-juelich.de/php/begin.php
I was told to post this here by the maintainer.

DeepMind team only shared the AlphaFold deep learning model instead of the full pipeline for model building, so there are many features are missing including input feature generation and 3D structure generation.
If you want to build a model for your protein, you still can try typical structure prediction servers such as I-TASSER, RaptorX, and so on. They are also pretty good. In addition, some of the servers (i.e., RaptorX-Contact) have their own contact/distance prediction methods and use them to build 3D models.
If your protein sequence has close homologs with known structures, then it would be better to use template-based modeling. CASP targets usually deal with hard cases, which do not have known homolog structures or have very distant homolog structures. So, for a typical structure prediction circumstance, I hope template-based modeling would be enough to get an accurate 3D model. (I do believe that classical approaches are still better on these relatively easy cases.)

I think we should give it a try even though google is not releasing all the code yet. Many research projects would profit from it.

As discussed in issue #18 the released code does not include Multiple Sequence Alignment (MSA) and feature computation for arbitrary sequences, though there we give some guide as to the features used for those accustomed to computing them.
But note that in any case the AlphaFold system does not use homologous structures (templates) it is focused on "Free Modelling".

Hi, here I have found a reimplementation of alphafold in pytorch: https://github.com/Urinx/alphafold_pytorch
Here the author provided almost all the rest of the pipeline including feature generation.
Still I could not figure out how it works, but seems one can figure out something from it:)


Fix typo in comment-on-pr.yml

I saw your comment on the GHA static hosting discussion and noticed this typo when browsing your code

Thanks!


Added conversions to tuples of up to size 12

Spoiler:

The R construct list() is a very special. It is hard to find an equivalent Rust type.
First, it is a contiguous data-layout, but it is the pointers, rather than actual data. The Rust equivalent could be Vec<Box<dyn std::any::Any>>. That is to say, it stores a list owned pointers, to any kind of data. I could make other Rust suggestions that would resemble list() better, but this one would do for now.
The issue is the heterogeneity of the stored elements. They do not have to be the same thing.
A fixed sized version of this is Rust tuples. A tuple of a certain size can contain any type, e.g. (i32, i32, f64). Is equivalent to the unnamed list that contains list(integer(), integer(), numeric()).
This PR adds conversions from list(...) to tuples. The tuple is of size 12, as this is chosen number by the std as well.
It is important to say, that if Robj is any other type, like integers(n=4), that does not mean the conversion would then target (i32, i32, i32, i32). Robjmust belist()/[List`](https://extendr.github.io/extendr/extendr_api/wrapper/list/struct.List.html). If you're looking for such a conversion, see #856 instead, as that is more appropriate.

[x] Things to do in a follow-up PR: All the extra functionality provided by impl_try_from_robj, such as Option<(T0, ..., Tn)>, etc. This was necessary to make the PR work with #[extendr]-macro, so it is also applied.

One interesting thing that I ran into using this is that it requires that the TryFrom<&Robj> and does not support when an owned TryFrom impl is provided.
Should we also support when TryFrom<Robj> is present?
impl TryFrom<&Robj> for Point {
    type Error = extendr_api::Error;
    fn try_from(value: &Robj) -> std::result::Result<Self, Self::Error> {
        let dbl_vec = Doubles::try_from(value)?;
        let x = dbl_vec[0].inner();
        let y = dbl_vec[1].inner();
        Ok(Point {x, y})

    }
}

/// @export
#[extendr]
fn sum_points(x: (Point, Point)) -> Doubles {
    let Point { x:x1, y:y1 } = x.0;
    let Point { x:x2, y:y2} = x.1;

    Doubles::from_values([x1+x2, y1+y2])
}

I already did that.. Right?

I understand now what you meant. One thing I've noticed about the extendr-api, is that it implements all conversions on &Robj,  and merely invoke the TryFrom<&Robj> method in TryFrom<Robj>-impl.
I don't think we can offer the flexibility that this particular impl requires TryFrom<Robj> instead of requiring the TryFrom<&Robj>. Furthermore, we #[extendr]-fn requires TryFrom<Robj>.
I've been contemplating adding another macro that can be used to assist with this boilerplate for users. But that has to be in follow-up PRs, and when I'm more confident as to what the constraints are.
@JosiahParry

"Furthermore, we #[extendr]-fn requires TryFrom."

Exactly, this feels like a weird differentiation. I'm not sure how to rectify it.
One thing that can be done—I think—is implement impl<T: TryFrom<Robj>> TryFrom<&Robj> for T or something like that?

It's a bit more convoluted. I am all for unifying things, but I don't see a path forward for this, at the moment.

i've pushed a change to add testthat tests to ensure that this conversion works with a TryFrom implementaiton and that the length error works as anticipated

Oops! Need to update snapshots.

Perfect! CI runs now, for the most part. MSRV is an error outside of this PR, and R-devel is problematic due to.. R-devel treatment of rust projects.


install_plugin action not found

That was quick :) thanks @stack72

Thank you so much! A great fix


Exception refactoring

Pull Request type

[ ] Bugfix
[ ] Feature
[x] Refactoring (no functional changes, no api changes)
[ ] Build related changes (Please run ./gradlew generateLock saveLock to refresh dependencies)
[ ] WHOSUSING.md
[ ] Other (please describe):

Changes in this PR
Refactor ApplicationException.Code to their own exception classes. This makes it easier to catch and process exceptions. Previously, ApplicationException is caught and the catch block contained a if-else block for checking the codes.

@jxu-nflx The PR has a large footprint. It might be easier to review one commit at a time.


Fix input handler being silent when siddhi app is not running

Purpose
$subject
Security checks

Followed secure coding standards in http://wso2.com/technical-reports/wso2-secure-engineering-guidelines? yes
Ran FindSecurityBugs plugin and verified report? yes
Confirmed that this PR doesn't commit any keys, passwords, tokens, usernames, or other secrets? yes

do we need to have compiled and paused, can't we just use the paused one?

@suhothayan Fixed in 956cf97


ARROW-12325: [C++] [CI] Nightly gandiva build failing due to failure of compiler to move return value

Gandiva build failure caused by failure of GCC 4.8.2 compiler to automatically move return value.  This is rather troubling.  I'm not super happy with the fix because it is not needed (and potentially non-optimal because it can prevent RVO) in later compilers and looks weird.  I'm open to suggestions.
Failed build: https://github.com/ursacomputing/crossbow/runs/2303374510
Godbolt showing I'm not crazy: https://gcc.godbolt.org/z/x138zevhE

I'm open to suggestions

Maybe add a comment to this effect so that whenever we remove support for gcc < 4.9, we can remove this? Or could you even use an #ifdef or something to only do this on gcc < 4.9?

I agree, an ifdef could be OK, though right now this is only used in tests.
I submitted https://github.com/lidavidm/crossbow/actions/runs/748817979 for now.

I'll side with @cyb70289 and @lidavidm that an ifdef would probably be overkill.  Feel free to merge.

ifdef looks more clear to show our intention.

@cyb70289 I updated line 95 and fixed up the commit message.  @pitrou is correct that we do not need to worry about RVO if the output type differs so it should be harmless (i.e. no need for an ifdef).

Can we merge this to include this into 4.0.0 RC2?

The build failures appear to all be expected.  MacOS is failing due to LLVM (ARROW-12467).  C GLib & Ruby is failing on Windows with segmentation fault but was doing so before this (we should probably create a JIRA).  The JNI failure is ARROW-11633


[FLINK-4097] Cassandra Sink throws NPE on closing if server is not available

Jira Issue

found a minor formatting issue, otherwise +1

merging


Consider using dead-key rather than Unicode combining characters to produce accented characters.

Currently, the layout uses Unicode combining diacritics to implement various special characters. I think this is incorrect and the norm specifies that the behaviour of those keys must match the current behavior of the ^ key on current AZERTY keyboards.
This means that, say, ^ must be a dead-key, instead of being an active key that produces a Unicode combining character.
For instance, although directly available from the keyboard layout, the é (LATIN SMALL LETTER E WITH ACUTE ACCENT) can also be input using the following combination: e, AltGr+é from the layout in the current version of the source code.
I recommend to make use of dead-key instead.
Therefore, with my proposition, typing the é character would be possible using the combination AltGr+é, e.
The difference lies in the order in which the keys are typed. For a dead-key, the user must first type the desired accent using the appopriate dead-key, and then the key corresponding to the character to be modified. This is consistent with the behavior of existing AZERTY keyboards.
On the other hand, using a Unicode combining character allows for greater flexibility because one is able to produce characters that would normally not be available. For instance, using combining characters, one would be able to type a ẽ́ (LATIN SMALL LETTER E, COMBINING TILDE, COMBINING ACUTE ACCENT).
I believe one must strive for simplicity and match the behavior of existing AZERTY keyboards. Therefore, my proposition is to use dead-key instead.

You are right about that.
Initially I only used Unicode diacritics because it was the easier way to publish a version quickly.
Of course, dead-key are the real expected behavior, but as it will require a lot of work, I decided to publish a rough version, then implement the dead-key combination after that, or let people contribute to the project to add them.
We can let this issue open, or create a new one as a checklist with all dead-keys of the layout, or even create one issue for each dead-key.
Feel free to suggest a solution if you want.


Markdown tables not supported

Hello again,
I noticed that when I ask the model for a markdown table the output is in text and it doesn't format to a table structure.
Example:
Here is a table in Markdown format with the top 5 countries and their fictional security levels:

"| Rank | Country     | Security Level |
|------|-------------|----------------|
| 1    | Switzerland | Very High      |
| 2    | Denmark     | Very High      |
| 3    | Finland     | Very High      |
| 4    | Sweden      | High           |
| 5    | Norway      | High           |"

the output should be like an actual table (visually)

I followed the exact same steps, but still, the output of table is raw text and not formated.
generate a markdown table of 10 most secure countries and their level of security
Here's a markdown table of the 10 most secure countries and their level of security, based on the 2021 Global Peace Index:

| Rank | Country | Level of Security | |------|-----------------|-------------------| | 1 | Iceland | Very High | | 2 | New Zealand | Very High | | 3 | Denmark | Very High | | 4 | Portugal | Very High | | 5 | Slovenia | Very High | | 6 | Austria | Very High | | 7 | Switzerland | Very High | | 8 | Ireland | Very High | | 9 | Czech Republic | Very High | | 10 | Canada | Very High |

Please note that the security levels are subjective and can vary depending on the source. In this case, I've used "Very High" for all top 10 countries to signify their strong performance in the Global Peace Index.

It works like a charm, thanks @phookycom
I hope you can help me with the previous issue too 👍🏼


Bugfix/macos encoding detection

When detecting the encoding, the .utf8 is being passed to the configuration, although the right encoding is the .macOSRoman

Thanks


Move database from firebase realtime database to cloud firestore

took about 11 months but looks like its done 😄


Request for advice: Easiest way to get balance of entire wallet

Is there any function to make it easy to get the balance of a wallet from a seed, pubkey, prvkey, etc?
Or should I just write my own function to derive the first (variable) addresses and iterate through them and check the balances and add them up?
Is this a function that would be useful within Bitwasp?
In theory, it would accept the seed or pubkey or prvkey, and the number of derived addresses to check, and an optional starting point to allow for a sort of pagination.
I have noticed a lack of wallet related examples in the examples directory, unless I've missed some.

There's no support for it at the moment - it implies the code has access to the full UTXO set. API's are an option, but since they're down to taste, I leave that decision to consumers.
If you're looking for something reasonably small, maybe write an app to listen for ZMQ notifications of new blocks! Parse each block for payments to your output scripts, and spends of the same. You could set a callback for both events for review. (Payments are always interesting, and depending on your app you might want to check the spend transaction as well)
There is an effort here to get a full node running in PHP: https://github.com/Bit-Wasp/bitcoin-php It gets very slow as the UTXO table gets full.. I've never had it fully sync to date. I'll probably migrate that table to leveldb if it makes it useful..

As for adding support, I'd prefer to keep this a purely offline library, and work with persistent data in a repository specific for wallets. We could then expose a WalletInterface and let people load them however they please!

I'm having trouble using RpcFactory::bitcoind to execute listunspent
Sample code:
<?php
require "vendor/autoload.php";

use BitWasp\Bitcoin\Address\AddressFactory;
use BitWasp\Bitcoin\Rpc\RpcFactory;
use BitWasp\Bitcoin\Bitcoin;

$network = Bitcoin::getNetwork();

$host = getenv('BITCOIN_RPC_SERVER') ?: '<IP_ADDRESS>';
$port = getenv('BITCOIN_RPC_PORT') ?: '8332';
$user = getenv('BITCOIN_RPC_USER') ?: 'rpcuser';
$pass = getenv('BITCOIN_RPC_PASSWORD') ?: 'rpcpassword';

$bitcoind = RpcFactory::bitcoind($host, $port, $user, $pass);

$addresses = [
  AddressFactory::fromString('1Mu1NPTTwx9hY2Ehfhs4u665RGvuohZA4f', $network),
  AddressFactory::fromString('129TQVAroeehD9fZpzK51NdZGQT4TqifbG', $network),
  ];

print_r("Block Height: " . $bitcoind->getblockcount() . "\n");
print_r("Array of addresses to check: \n");
print_r($addresses);
print_r("\n");
print_r("Response: \n");
print_r($bitcoind->listunspent(1, 9999999, $addresses, $network));
print_r("\n");

Which results in
Block Height: 409750
Array of addresses to check:
Array
(
    [0] => BitWasp\Bitcoin\Address\PayToPubKeyHashAddress Object
        (
            [hash:BitWasp\Bitcoin\Address\Address:private] => e5387418a3614cba40e6de0c1c782b24433ddf17
        )

    [1] => BitWasp\Bitcoin\Address\PayToPubKeyHashAddress Object
        (
            [hash:BitWasp\Bitcoin\Address\Address:private] => 0c917ec56ded601852e4caaa37d7bf9643773730
        )
)

Response:
Array
(
)

As you can see, I can receive getblockcount so bitcoind RPC is working just fine, but when I pass the array of address objects to listunspent, I'm not getting anything in reply.
Any suggestions? I'm not sure how to determine if the problem is with BitWasp\Bitcoin\Rpc\Client\Bitcoind::listunspent or with my bitcoind.
root@bitcoin18:~# bitcoin-cli -version
Bitcoin Classic RPC client version v<IP_ADDRESS>-unk

Nevermind, it was my mistake. I thought that you could just enter an arbitrary address and it would return the information about it. I didn't realize that you have to import the address into bitcoind before it'll process the listunspent command.


feature: editor lazy loading

recheckcla

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Warn if garage door is opened when not home

If the specified presence sensor is not home and the garage door is opened, send a push notification.

This is handled by Smart Home Monitor, no need for it in app


[catch2] Use json to save temporary information when testing

When running locally, on Docker with GCC5, I had the follow error:
ERROR: catch2/2.13.7 (test package): Error in generate() method, line 37
	with open(self._todos_filename, "w", encoding="utf-8") as file:
	FileNotFoundError: [Errno 2] No such file or directory: '/home/conan/project/test_package/test_output/build/generators/catch2_test_to_do.yml'

It occurred because the todo_filename's directory does not exist when running generate, so we would need to run a mkdir first.
On this PR I updated few things:

self._todo_filename is not needed, as the variable is not re-used indeed. It's lost between conan test execution
Python does not offer native YAML support, instead, we got a ride from Conan dependencies, but it's too risk. On the other hand, JSON is supported by python itself and we run no risks.
Using Conan tools to save/load not only simplifies some lines, but also may consider permission tricks on Windows for instance.
Move temporary file to build folder, as it can be removed after building.

[ ] I've read the contributing guidelines.
[ ] I've used a recent Conan client version close to the currently deployed.
[ ] I've tried at least one configuration locally with the conan-center hook activated.

Because someone keeps changing the client !!!!!! Grrr


refactor: addition of codeflare components as scrape targets and availabil…

…ity alerts

Adding of kuberay, codeflare and mcad components as scrape targets for the prometheus deployment.
Addition of up alert for the each component pod.
The SOPs for each component are in this PR
MCAD and CodeFlare components need to expose metrics endpoint before the alerts are passing.
With this PR, Prometheus is actively looking for the pods and the alerts fire.
DO NOT MERGE UNTIL WORK IS CARRIED OUT IN MCAD AND CODEFLARE
Description

How Has This Been Tested?

Tested on an OSD cluster with the RHODS addon.
Kill the kuberay-operator pods, and watch the alert fire in the prometheus UI. (don't forget to scale down the codeflare and rhods operators)
Once these changes are there, the prometheus deployment must be restarted.

Merge criteria:

[ ] The commits are squashed in a cohesive manner and have meaningful messages.
[ ] For commits that came from upstream, [UPSTREAM] has been prepended to the commit message.
[ ] JIRA link(s):
[ ] The Jira story is acked.
[ ] Live build image:
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work.
[ ] QE contact acknowledges that this has been tested and is approved for merge.

/hold

@dimakis there's something weird with the commit history in your PR, did you branch out of main?

@dimakis there's something weird with the commit history in your PR, did you branch out of main?

ah I messed up and accidentally committed deadman snitch and PD keys, so I had to go through the steps that SRE gave me to change the history

If I understand this PR, this PR will check the status of the MCAD application, MCAD is launched via KAR controller, the question then would be how does the App know if it's working or not?

If I understand this PR, this PR will check the status of the MCAD application, MCAD is launched via KAR controller, the question then would be how does the App know if it's working or not?

Yes, it'll just check the availability is all really. It's the bare minimum SRE requirement for release. I've got a doc which outlines a more detailed SLI/SLO metric plan to be introduced later but we're just starting at the basic level to implement then building on top of that.

I added a commit to this PR with additional rules involving SLO alerts for MCAD and CodeFlare Operator. You will find several probe_success:burnrate recording rules and alerting rules. Added 3 alerts for each component based on the error budget burn rate. These were based off the way Data Science Pipelines Operator has defined them.
The importance of these alerts is to ensure the high availability of these components by tracking the rate at which they are failing to meet their SLOs. This is assuming these components should be available and operational 99.95% of the time.
@astefanutti I'm very interested in your thoughts on this and/or changes to be made? Thanks in advance.

Moved to here: https://github.com/red-hat-data-services/odh-deployer/pull/380


Add EulerA scheduler

Currently is borken. Would appreciate feedback.

@jdluzen Can you share the model you are using?

@jdluzen I was able to get the euler scheduler working on my model. Can you test on your model and let me know if it works for you?

It works! UINT8 quantized too! Did I port the wrong file?

Yay!! I am going to refactor the scheduler a bit in a new PR to main because I am also working on an EP config in main rather than having branches for each EP.
You didnt port the wrong file but there were issues in the math. Python and PyTorch handle some math operations and random generation differently than C# so it needs to be implemented different to work correctly.
Thanks!!


Add support for Pulumi

From DevBlogs:

Please upvote if this is important to you!

Pulumi currently supports a YAML authoring language, which is convertible to a fully fledged programming language.
Being able to output Pulumi YAML, then running pulumi convert would be awesome

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI  is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ...  So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI  is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ...  So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

Yes, another related benefit is to be able to share templates that contain the application and the infrastructure code. Pulumi CLI is only about the infrastructure code. For people getting started using Azure, I believe it's nice to have some boilerplate (infra+app) to build an application that can be directly deployed to Azure.

Why do you think the code that pulumi creates with its cli is not sufficient? I am just curious.

It's not that pulumi cli is not sufficient but about being able to use azd with pulumi. Azure Developer CLI  is not only about Infrastructure as Code but also about generating the application code, generating the pipelines to deploy the code, deploying the application, and monitoring the application, ...  So for someone who wants to manage its application using azd (code, build, deploy, monitor) but with the infrastructure code using Pulumi, support for Pulumi in azd is a must have. Or maybe I misunderstood the question and your question was about what @jaxxstorm said?

I can see the benefit in that if everything is generated automatically with 1 command thus saving time. It was not related to @jaxxstorm comment.

Yes, another related benefit is to be able to share templates that contain the application and the infrastructure code. Pulumi CLI is only about the infrastructure code. For people getting started using Azure, I believe it's nice to have some boilerplate (infra+app) to build an application that can be directly deployed to Azure.

Yes defintely. I was actually working on building a SaaS template for azure some time ago so I can see the benefit of that.

@savannahostrowski does roadmap mean this is planned? Any details?
It would be great if devs could use .NET Aspire, which enables automated deployment via AZD, together with Pulumi. I'm working on a project where I'd like to do that.
So far I have not seen concrete advice on how to combine / plans for supporting that in the bits - see discussion on pulumi repo

It would be great if devs could use .NET Aspire, which enables automated deployment via AZD, together with Pulumi. I'm working on a project where I'd like to do that.

I'd like to better understand the scenario here - specifically what "together with Pulumi" ends up meaning.  Do you simply want azd to use pulumi to deploy the infrastructure for your Aspire app behind the scenes?  Are you trying to combine the infrastructure from the Aspire application with infrastructure managed directly by a Pulumi program?
I suspect for the former - just wanting to use pulumi to do the deployment of the infrastructure that your Aspire application needs - so you get like a Pulumi stack you can see in the console, can leverage esc or other stuff - is something we'd be able to support (there is the interesting issue that you point to about how today we generate bicep and for Pulumi we'd need to either generate something else or use a converter to convert the bicep to something Pulumi understands).
To be frank, the second case is harder - we currently don't have a great story for using generated infrastructure from Aspire together with hand authored infrastructure for other resources - we'll need to come up with a solution for this independent of support for Pulumi (since azd wants to support a polyglot environment) but we haven't thought about the design in great detail yet.

I'd like to better understand the scenario here

Thanks for asking @ellismg. I don't have Pulumi experience yet so I appreciate any insights / advice from those who do - it's fortunate that you have worked there so I'm interested in your thoughts.
My main goal is to simplify Azure distributed application development for C# developers, and to enable them to use their core C# skills + tools in as many areas as possible.

Aspire serves that goal very well (especially when combined with MS Orleans). However while with the AZD support Aspire extends into Azure IaC to a degree, Aspire is not a full fledged IaC solution.
Pulumi also fits very well, and is a mature and complete IaC solution with (amongst others) C# support for Azure.

What I imagine AZD Pulumi support to enable:

From an Aspire user perspective, you could just supply additional parameter(s) to the azd up command to make that infrastructure a seamless part of what Pulumi creates and knows about (so Pulumi tools can be used for that stack).
A common case is that some Azure resource type is not (yet) supported in Aspire but it is a seamless part of the application infrastructure (e.g. an Azure SignalR resource). I would like to be able to create that resource in a Pulumi program, and use e.g. a connection string for that resource in the application.

Would the above be possible with the first approach you described?

for azd to use pulumi to deploy the infrastructure for your Aspire app

(I suspect so because no direct Aspire - Pulumi integration is needed; the application code just uses resources defined by both)

Another case (not encountered yet but I only did Aspire POC's so far) is that you need more control of the Aspire defined resource than Aspire enables. I suspect that would require the second approach you described? A fallback would be to just make a resource definition either fully owned by Pulumi or by Aspire, not both.

In summary, if my reasoning above is correct the first approach you described should be sufficient, while the second polyglot approach seems a more powerful option further down the road.
Does this make sense from your perspective? If you see limitations or other options I'm also interested - thanks!

@VincentH-Net
These are really great questions and really get to the heart of some of the problems we're trying to solve as continue towards a GA of Aspire and improve the azd support for it.
As you mentioned - I worked at Pulumi for a while and have a pretty good idea about what a story in Pulumi might look like (granted the team over there has done a ton of amazing work since I have left, but I feel like I still have a pretty good handle on the model).  I think the answers to these questions depends on if you are looking at the problem through a "pulumi first" lens or an "azd first" lens.  Since I work on azd now - I'll try to answer these questions here from the azd viewpoint, but at some point I will also try to go to the Pulomi discussion and add my thoughts as a guy who had his Pulumi hat on instead of his azd hat.

However while with the AZD support Aspire extends into Azure IaC to a degree, Aspire is not a full fledged IaC solution.

I think this is a bit of a philosophical question - my general view is that Aspire is/can be this but there are just some gaps today in the end to end that we need to go improve.  You've outlined some of them:

The Aspire.Hosting.Azure package provides a set of Aspire components for working with Azure resources - but there is not complete coverage for the Azure platform.  Many popular services do not have .AddXYZ() methods to be used on the DistributedApplicationBuilder type.  Because of this - you can't use Aspire today to describe the complete shape of your distributed application if you need these services.
For the types that are in Aspire.Hosting.Azure today there are not a complete set of knobs to control every setting on the service you want to control.  For example, there's a component that allows you to create an Azure Blob Storage Account but there are not knobs you would need to configure properties (for example, imagine you were trying to use it to store and host the assets for a static website - the component does not give you everything you need to control say the custom 404 page, leading you to need to use the az CLI or the Portal or something to control these knobs after the application has been deployed.
The story for "instead of spinning up this resource, I want you to connect to an existing one that has already been provisioned by someone" isn't smooth yet.  This means even if you use some other IaC program like terraform or pulumi (or even just write straight bicep or arm templates and us az to deploy them) using these resources from your Aspire app can be challenging.  Also if you want to do something like, in the Development environment I provision an database but in the Production environment I connect to an existing database and have a great end to end experience things are really rough today.

We've been talking about how we support all these cases.
For (1) we're looking at how we can auto-generate these Aspire components in the same way that the Azure SDK team generates much of the code for the .NET Azure SDKs.  We're currently in the ideation stage of this work stream, but we have some really good ideas and I suspect over the next few previews you'll start seeing stuff come online here.  We're also looking at the ability to do something like builder.AddAzureDeployment("mycustomresource", <path-to-a-bicep-or-arm-template>) which will allow you to insert arbitrary Bicep or ARM templates into your Aspire application, which azd will deploy with the rest of the infrastructure of your application.  I expect that this primitive is the thing that all the auto-generated code will build upon.  My guess is that starting around preview 4 you'll see support for this primitive.
For (2) We're currently thinking about a strategy that allows you to add annotations to resources in your app host and tools like azd can interpret these annotations to control certain parts of the resource.  For example, today when you do builder.AddPostgres() to add a Postgres database to your app and deploy it with azd, azd uses the Postgres dev service feature in ACA to give you a database.  That's great for development because it spins up fast and is cheap to run.  But in production you might rather use something like Azure Database for PostgreSQL - Flexible Server to offload the operations of the server to managed service so you don't have to play SRE (and you get better performance and durability).  Currently we are thinking that might look something like this builder.AddPostgres().WithMetadata("azd-postgres-server-type", "flexible-server").  From this loosely typed interface primitive we could expect building a strongly typed interface over this.  This would still not expose every knob of every service - and so we are also thinking about things like "can you give us a diff to apply to the object that we'd send to the ARM control plane to create the resource.  It's less clear to me right now how that would work - but we do know this is an issue and are trying to figure out a solution.  We are also thinking about how to improve infra synth to allow to you just create IaC for parts of your application (e.g. for just a few resources) so you can hand edit these parts and let azd + Aspire auto-generate and handle the rest of stuff.
We also have been talking about (3) and it's something we want to deliver as part of Aspire GA.  We have some ideas here and you should see that work come online during the preview 4 timeframe as well.
All of this work can be done without adopting Pulumi and we plan to build all this stuff on top of Bicep.
A place where Pulumi would be helpful is if you wanted to start to manage resources in non Azure clouds (since azd will always be focused on Azure) or you cared about using Pulumi instead of the ARM Deployment (perhaps you want to be able to use esc to manage configuration, perhaps you want to buy into their Policy as Code story, or use their RBAC support for organizations to control who can deploy to what stacks).  For us to make this work we'd have to two major things:

Add support to azd provison to use Pulumi instead of ARM to do a deployment.  This would be similar to our (beta) support for using terraform with azd.
Allow azd to generate a Pulumi program to describe your Aspire application and hand that to pulumi during azd provision.  We could either do this by modifing azd to also be able to generate Pulumi code in addition to Bicep or do something like generate the bicep and then run a converter to turn it into Pulumi.

This is a large amount of work (and adding another IaC provider to azd has some higher than expected costs, since IaC provider ends up having impacts on the rest of the system and represents a non-trivial cost to maintain over time) and so as a small team right now we are focused on really trying to nail the "Azure Native" version built on top of Bicep.  We also selfishly want to use it as a way to highlight places where we can improve the developer experience of folks who work with Azure day to day so we can drive improvements to the platform.
If I was to put my Pulumi hat on I think the answer would be something like: "There should be a Pulumi component that let's you do something like: new AspireApp("<path-to-app-host-project>"); and the implementation of that component talks to the app host to extract its manifest and then "compiles" that to a set of Pulumi resources."  If I was still on the Pulumi team, that is probably what I would be looking at building.
But that's a discussion for another day and another issue (I will try to find some time to brain dump thoughts into that discussion thread at some point).
Hope this is helpful and gives you an idea about what we are thinking about and "where the puck is headed".  Sorry for taking a while to get back to you - you're touching on the some of the core complex issues we've been struggling with and so it took some time for me to synthesize our current thinking and lay it all out for you (while at the same time trying to make azd better and work through the design meetings around these issues).
Hope this is helpful - happy to answer any follow ups, but it might take me some time :-).

Thank you @ellismg - I really appreciate the effort to share this comprehensive overview. It took some time for me as well to let it sink in and reciprocate. The examples you mention match my experience; sharing this is very helpful in wrapping one's mind around the technologies, and it can help to determine a course in solution technology roadmaps.

my general view is that Aspire is/can be this (= a full fledged IaC solution)

Hmmm... my grok (so far) is that Aspire helps simplify multiple aspects of distributed application development (IaC being one), while Pulumi is fully focused on IaC. Think e.g. about all the different environments a solution exists in with their respective infrastructure variations and partial IaC reuse across them, versus just two infrastructure variations: local and deployed.
As a .NET architect I like to have as much power as possible, as soon as possible, with early maturity, to reduce complexity and effort. A clear product focus helps for that; from that perspective it would appeal to me to have Aspire become truly great for development-centric aspects of distributed applications, including the first step / core IaC, and focus on a smooth and flexible integration with specialized infrastructure solutions such as Pulumi, rather than trying to make Aspire a full IaC competitor. As seen on many occasions, combining strengths and clear focus makes an ecosystem more valuable than adding more alternatives that have a bigger overlap with a less intense focus. It results in earlier maturity as well.
Just food for thought... not disagreeing with you here.
That being said, the Azure focus of AZD is clear and I think some of the features described above would be great:

builder.AddAzureDeployment("mycustomresource", <path-to-a-bicep-or-arm-template>) seems like an excellent first step. Generating full Azure coverage for Aspire as well. Preview 4 (of Aspire) is quite close too!

builder.AddPostgres().WithMetadata("azd-postgres-server-type", "flexible-server") this one raises some concerns - it seems hard to discover and debug, hard to support in IDE, and has the potential to create the type of "magic configuration strings" pain that plagues today's distributed application development tooling, and that Aspire set out to solve in the first place. This also touches upon my "Hmmm" above - perhaps this would be better served with providing a frictionless handover to specialized IaC? That could be something the Aspire/AZD teams build, but it could also be an active collaboration with Pulumi et al to stimulate and help them use Aspire as input.

I wonder which would be a better integration point - deployed infrastructure or the template / manifest / program that created it? Is the goal to support manually deployed / (partly) configured resources?

In short, I love the plans in 1 to address gaps 1 and 2 - I will be using that functionality for sure. Plan 2 I would not do and instead try to smooth integration (incoming and/or outgoing) with other IaC products. That might also obviate the need for plan 3.
Hope this helps - interested to continue the evolution of thinking, and thanks again for sharing!

My point at

Hmmm...

above prompted me to start a wider discussion about IaC in the Aspire repo; afaiac we can continue with AZD specific things here as well. Hth

Quick reference to the Radius project: https://github.com/radius-project/radius
Radius ultimately provides a separation of concerns between your infrastructure definition and the app workload.


Reordered port numbers for backend services

[x] Explain what this PR does

Fix something that was bugging me (Joshua)

[x] Documentation updated (if needed)

n/a

[ ] Code reviewed by someone else on the team
[ ] PR is ready and can be merged (do not check this until all other boxes are checked)

@Lmd117 actually, I realized that base URLs have to be updated completely to match the microservices architecture. Since this hasn't been done yet, it doesn't make sense to hold off on merging this branch. We can deal with updating the URLs in a separate PR (probably one that builds on top of this one). I will go ahead and merge this one.


fix(package.json): update whatwg-url version due to deprecation warni…

In addition to resolving the problem addressed in issue #30, the @types/whatwg-url package has no place in depedencies and has therefore been updated in devDependencies.

the @types/whatwg-url package has no place in depedencies

See https://github.com/mongodb-js/mongodb-connection-string-url/issues/15 – we can’t just drop the type dependency like that, unfortunately.