Run the screen on SPI1.

Hello,
I love this driver. It's fast, featured, and customizable. However, it's missing a feature I'd love to be able to use: Having the screen on the SPI1 auxiliary line rather than on SPI0. I have almost no C experience and I was barely able to customize the overscan settings in gpu.h, so I'm definitely not the guy for the job in this case, but certainly it would be very helpful for those with hardware limitations or who want to use multiple screens for an expert to code in this ability. I'm aware that the auxiliary SPIs are slower than the main one, but with a driver as fast as this one, it shouldn't matter - anything above 10fps is acceptable. I'm sorry if this is glaringly impossible or an obvious fix, but it's a problem I and many people in threads I've encountered in my research have faced, to little avail.
Sorry if this isn't really constructive or if this isn't the right place to post, or if this question has been answered many times and I'm just stupid. It would be heartily appreciated, especially if in the form of a tag in config.h.
Thank you so much,
Sam

Unfortunately the issue with aux SPI is not just the screen refresh rate, but the CPU utilization. The auxiliary mini-SPIs SPI1 and SPI2 do not have DMA support, so achieving frame rate will cause major CPU usage. You can simulate this with -DUSE_DMA_TRANSFERS=OFF CMake flag, and observe in top how CPU usage goes up to 100%. Maybe with 10fps it would be reduced, but still likely a good 30% of the Pi's CPU headroom. That is why developing such a general SPI support has not been a particularly interesting project.
If someone would like to contribute such a general SPI support, I would be happy to merge that in though.

Ah, that makes sense. Thank you.


proposal: Go 2: allow explicit conversion of bool to arbitrary numeric types

In some cases, being able to do arithmetic or bitwise operations in which you can use a true-or-false value without a branch is highly appealing. It would be convenient to allow conversion of true and false to arbitrary integer types, producing the values 1 and 0 respectively. Conveniently, unless I get around to writing up my int0 proposal and convince someone that it's a good idea, every numeric type in Go can represent both of those values.
I've been told that the Go compiler is already smart enough to generate reasonable code for something like this:
int x
if cond {
  x = 1
}

However, even if the compiler can do it, it's a lot more code to read than something like int(cond) would be, and whether or not it's branchy is less obvious and I don't know whether it's safe to use this in code where a branch is Too Expensive. (Unfortunately, Google Groups has destroyed any once-functional Usenet searching, so I can't find the reference, but I seem to recall someone getting a 20% overall improvement in a PRNG by replacing if (z) { x += y } with x += y * z.)
I originally strongly disliked the thing where true and false weren't numeric values, but I've come to quite appreciate the need to make a conversion between booleans and numbers explicit. But "explicit" need not mean "verbose". Allowing int(a < b) to be 1 or 0 depending on whether a was or was not (respectively) less than b seems like it would simplify a lot of code without hurting anything.

Previous discussion at #9367 (and other discussions before that, I think).
Not sure it has ever gone through the modern proposal process, though.

I intentionally declined to propose converting integers to bools. I want to be able to bitwise-or bools and otherwise integrate them into integer expressions, but we already have != 0 if we want to go the other way, and it's clear and short.

(@seebs if you're responding to me, proposal #9367 is about converting bools to numeric types, just like this one.)

One of the comments in it cited to the ambiguity of what bool(intValue) should mean as a weakness, which I agree it sort of is.

Slightly off-topic but re the discussion in #9367, to be honest, the whole "true could be interpreted as both 0 and 1" argument seemed weak to me. The only example provided was exit statuses across operating systems, which is platform-specific anyways. For example, VMS treats 0 as a warning and 1 as a success, while POSIX treats nonzero as an error.
Explicitly converting between the zero value of bool to the zero value of int is predictable behavior. It also has the benefit of matching other languages' interpretation of a bool -> int conversion.

and whether or not it's branchy is less obvious and I don't know whether it's safe to use this in code where a branch is Too Expensive.

As far as I understand the proposal at least two reasons are given in favor of the proposal:

knowing if there is a branch or not
The Go spec does not mandate how bools are represented internally and the proposal does not force an implementation and therefore the implementation could use a branch with the same performance as the if version before. Which means the current situation is that an if might suggest there is a branch but the compiler may be smart enough to avoid it while with the proposal it is not clear from code there might be a branch and then there actually could be a branch used by the implementation.

performance
If performance is to be made as an argument then I think the proposal also needs the requirement that the conversion needs to be (or can be) faster or as fast than equivalent if version. However I would not think the performance or branchness requirement should be part of the language spec.

As noted in practice there likely isnt a performance benefit as the current compiler already optimises the existing pattern to convert a bool. If there is we can work on fixing those without changing the language.
Alternative Suggestion:
Instead of changing the language the std lib could contain a generic function that converts any integer type to bool and we make sure at least the Go GC implementation always inlines and optimises the if pattern in that generic function to branchless and performant on architectures that allow that.

I mean, it could be a function.
but is there any reason for which, I dunno, math.BoolAsInt32(true) is a better spelling than int32(true)? I seem to recall that, at least at the moment, function calls imply at least a nop even when inlined to allow the compiler to distinguish what code is executing... But conversions, when they're allowed at all, are also allowed to be completely free.
The primary goal here is to make the code more expressive and allow short but still-reasonably-clear code, but also to hint that this shouldn't be a branchy operation.

but is there any reason for which, I dunno, math.BoolAsInt32(true) is a better spelling than int32(true)
The difference is that a language change has a much higher bar to be acceptable then adding a new function in the std library. And a language change likely also doesnt have the upside to guarantee branchness as that is an implementation detail of the specific Go implementation.

but also to hint that this shouldn't be a branchy operation
Just because it is short and doesnt contain an if is I dont think a good hint whether it branches. e.g. other conversions string(int) also involves branches and look the same from a syntax complexity.

Instead of changing the language the std lib could contain a generic function that converts any integer type to bool

I was thinking this while I read the thread. Having to explicitly call a math or math/bits function would also mean that it would be harder to convert a boolean to an integer by accident.

This proposal does seem very similar to #9367 and #6011.  What has changed since then?
I'll note that it would seem odd to support converting from boolean to numeric but not support converting numeric to boolean.  Are there any other one-way conversions in Go?  (Other than converting to an interface type, for which the reverse conversion requires a type assertion.)

@ianlancetaylor it makes it a lot easier to read branch free code, when you can use the normal algos instead of having to write things in asm.
It helps with math code when you can do a multiply by 1 or zero instead of having to add 5 or 6 extra lines and break up flow with a conditional.
@mvdan that would be a bit of a pain to read the code.
Having to do math.boolToNumeric(uint)(a<b) in a line with a lot of math is a lot more awkward than just uint(a<b)

The compiler defines:
func b2i(b bool) int64 {
	if b {
		return 1
	}
	return 0
}

Which we then use lots of places. So although I think converting from bool to int is useful, it doesn't appear to me to require language changes - if you've only got a few instances, then math.boolToNumeric[uint](a<b) seems easy enough, and if you have lots of instances making a helper function to reduce repetition is also easy.

@randall77 that defeats the purpose by introducing a branch (or similar flow control) and possibly a function call.

@docmerlin Please don't confuse a branch in the Go code with a branch in the generated machine code.
For example, when using the gccgo compiler, this Go function
func Cmp(a, b int) int {
	if a < b {
		return 0
	}
	return 1
}

compiles to this assembly code (ignoring the stack-splitting code):
	xorl	%eax, %eax
	cmpq	%rsi, %rdi
	setge	%al
	ret

For cryptographic code it would be great to have a function to go from bool to int in guaranteed constant time. Maybe it can go in crypto/subtle along with all other constant time functions.
Packages like filippo.io/edwards25519 have to use int instead of bool, leaving all values other than 1 and 0 undefined, so that they can be used as inputs for crypto/subtle functions without branches.
We'd probably also need the other way around, because those values are also generate in constant time by masking and bit shifting.

Ohhh. I hadn't thought about the constant-time guarantee, but that does seem relevant. Although... Is the naive CPU code actually constant-time? I have no idea.
It seems like it'd be nice to be able to drop the extra nop, although it probably doesn't matter much.

This is a side comment, but the notion of guaranteed constant time in non-assembly code appears to require that the compiler never insert branches that aren't present in the source code.  I doubt any compilers do that today but it's hardly impossible.

Every cryptography engineer expects in fear the day that will inevitably happen, and secretly hopes to retire before then. There are occasionally rumors of LLVM-related sightings. A much more sinister prospect than quantum computers, which at least will require new exciting designs, it's considered indelicate to talk about it in polite cryptography circles.

Well, now I know the next GCC optimization that I should work on!

ಠ_ಠ

FWIW, neither performance nor knowing about branching are my reasons for supporting this proposal. I just think that some algorithms are easier to read when, for example, using the truth value of some decision procedure later in some function isn't obscured by control flow.

FWIW, neither performance nor knowing about branching are my reasons for supporting this proposal. I just think that some algorithms are easier to read when, for example, using the truth value of some decision procedure later in some function isn't obscured by control flow.

Understood.  But it seems that b2i function like the one that @randall77 mentions above addresses that.

Although it's been implied, I don't think anyone's explicitly said that the b2i function mentioned above does in fact generate branch-free code. For example in this code the line sum += b2i(x) compiles to just the single instruction ADDQ    SI, BX.

What do people think of
func AsInt[T constraints.Integer](b bool) T {
    if b {
         return 1
    }
    return 0
}

    v := AsInt[int32](b)

@ianlancetaylor Are generics really pulling their weight in that example? It seems to me that just returning int and adding an explicit cast when necessary would be just as flexible and arguably slightly simpler and more readable.
func AsInt(b bool) int {
    if b {
         return 1
    }
    return 0
}

v := int32(AsInt(b))

@magical Fair point.

Based on the discussion above this doesn't seem to rise to the level of an appropriate language change.  Therefore, this is a likely decline.  Leaving open for four weeks for final comments.

No further comments.


SR data source

Just as you provide PIF data source, maybe it will be a good idea to provide similar for SR?

Definitely :). I just haven't had the time yet. I'll get this working soon.

Vote for this. Otherwise it's not possible to provision VM's to hosts with local-storage, without knowing all SR UUID's from somewhere. Or is there another option to accomplish this without a PIF Ressource for the SR UUID's ?

Has anyone found a workaround for this issue?

Sorry for the silence on this issue. I can probably set aside some time this weekend for implementing this.

@m0ps @dmcanally @MrOffline77 I have a PR open that has basic support for this. Right now it uses the Storage repository's name_label to identify it. You can see the example terraform code for it in the acceptance test I wrote here. I'll have docs soon but that's the best thing I have to show off how it works for now.
Would this solve your use cases? I'm thinking it would be nice to filter down based on the storage repository's type as well (lvm, local storage, etc) but I wanted to start with something basic and get your feedback first before I spent more time on enhanced functionality.

Since I haven't heard any feedback on my proposal in #31 I'm going to move forward with it. It is a decent starting point and extra functionality can be added later if it doesn't solve the needs that led to this issue's creation.

Apologies for the lack of response on this (work and all).
I was able to test and it is working as desired, so THANK YOU for the quick feature add.
`
data "xenorchestra_template" "template" {
name_label = var.template
}
data "xenorchestra_pif" "pif" {
device = eth0
vlan   = -1
}
data "xenorchestra_sr" "sr" {
name_label = "Local storage"
}
resource "xenorchestra_cloud_config" "xoa_cc" {
name     = "Bootstrap"
template = <<EOF
#cloud-init
...
EOF
}
resource "xenorchestra_vm" "xoa_vm" {
count        = 1
memory_max   =<PHONE_NUMBER>4
cpus         = 2
cloud_config = "${xenorchestra_cloud_config.xoa_cc.template}"
name_label   = "vm00"
template     = data.xenorchestra_template.template.id
network {
network_id = data.xenorchestra_pif.pif.network
}
disk {
sr_id      = data.xenorchestra_sr.sr.uuid
name_label = "vm00"
size       =<PHONE_NUMBER>4
}
`

Update: So I run 3 nodes in my xen pool. The current implementation will only worked for a pool wide SR. This will not work for leveraging "Local storage" sr. Each time terraform runs, data.xenorchestra_sr.sr.name_label will potentially be populated by a different node's Local Storage SR.

@dmcanally thanks for the response. Sounds like we need to add pool_id to the sr data source and even add a pool data source (so that we don't need to pass pool ids).
Reopening and I'll find time to give this attention soon.

I was able to work around this for now by giving each of the "Local storage" sr's in my pool a unique name (i.e. "Local storage 1", "Local storage 2", "Local storage 3" etc). I then pass that to xenorchestra_sr via a variable. When you push an update, I'll test as soon as I can.
Thanks,
Denver

@dmcanally I have just created version 0.3.2 which adds an optional pool_id attribute (PR #37). Please upgrade and let me know if this behaves as expected. The change is covered by acceptance tests so I am confident it should work well.
I have created #36 as a follow up item so that you won't need to have all of your pool IDs hard coded in the terraform files. I intend to fix that soon.

I'm going to close this again since #37  and #36 have both been implemented. Please let me know what you think when you give this a try.


Update several dependencies at once

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.509%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
3186

Relevant Lines:
3444

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 92.509%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
3186

Relevant Lines:
3444

💛  - Coveralls

@liamhuber I prepared a release, if this fits for you, I would release it on the weekend to get the versions of pyiron_atomistics up to date on the cluster :)

@niklassiemer thanks for handling the codependent updates!
Will it be a problem to hold until Monday? I have a big fix  PR ready to go, but am experiencing a pycharm-GitHub issue; I should be able to make a release by Sunday evening though

No, would be a-ok as well. I just wanted the dependbot PRs to fetch the update on Monday, such that I could have a new consistent environment including pyiron_snippets etc. by the next deploy to cmti on tuesday. However, otherwise it simply stays similar for another week ;)

Super, I'll get it out as soon as I can!


why x.unsqueeze(dim=0)

Hi,
Thanks for your great work and make it public.
I wonder why .unsqueeze(dim=0) is used here.  It seems that this is not a necessary operation.
Best

Hi thank you for raising it, we have removed the unnecessary part.


Avoid uint8 format in SchemaStore JSON Schema

The new tabSize property uses uint8, but SchemaStore rejects it. (I edited the generated PR manually.)

Yes good call.

https://github.com/SchemaStore/schemastore/pull/3034


Bug in user defined system.

The "desiredGNSSsystems" does not work for RINEX v2. It will always run on both GPS and GLONASS. After this is fixed in the reading routine, the None arguments have to be removed in readRinexObs function in readRinexObs module.

It now works


hardhat test fails with TypeError: nativeNC failed when hardhat-gas-reporter enabled

Regardless of what ever I tried, hardhat-gas-reporter fails always with nativeNC failed exception.
Made sure there is no eth-gas-reporter installed, and my hardhat config has only enabled: true
, and removed all other options like coinmarketcap, currency, token etc. to ensure they are not causing the exception.
The tests passes successfully (with network localhost and hardhat) if I comment out or remove require("hardhat-gas-reporter"), so tests are not the problem.
Tried with networks, hardhat, localhost, testnets (rinkeby, goerli), nothing helped.
Any help is much appreciated.
$ npx hardhat test --network localhost
An unexpected error occurred:
TypeError: nativeNC failed:
at createInvalidReporterError (D:\development\carbon\cct\node_modules\mocha\lib\errors.js:207:13)
at Mocha.reporter (D:\development\carbon\cct\node_modules\mocha\lib\mocha.js:343:17)
at new Mocha (D:\development\carbon\cct\node_modules\mocha\lib\mocha.js:196:6)
at SimpleTaskDefinition.action (D:\development\carbon\cct\node_modules\hardhat\src\builtin-tasks\test.ts:95:21)
at Environment._runTaskDefinition (D:\development\carbon\cct\node_modules\hardhat\src\internal\core\runtime-environment.ts:219:14)
at Environment._runTaskDefinition (D:\development\carbon\cct\node_modules\hardhat\src\internal\core\runtime-environment.ts:219:14)
at Environment.run (D:\development\carbon\cct\node_modules\hardhat\src\internal\core\runtime-environment.ts:131:14)
at SimpleTaskDefinition.action (D:\development\carbon\cct\node_modules\hardhat\src\builtin-tasks\test.ts:159:28)
at Environment._runTaskDefinition (D:\development\carbon\cct\node_modules\hardhat\src\internal\core\runtime-environment.ts:219:14)
at Environment._runTaskDefinition (D:\development\carbon\cct\node_modules\hardhat\src\internal\core\runtime-environment.ts:219:14) {

code: 'ERR_MOCHA_INVALID_REPORTER',
reporter: 'D:\development\carbon\cct\node_modules\eth-gas-reporter\index.js'
}
$ npm list | grep gas-report
├──<EMAIL_ADDRESS>$ npm list -g | grep gas-report
├──<EMAIL_ADDRESS>

I got a similar error message! Did you find any solution?

+1
[ERR_MOCHA_INVALID_REPORTER](code: 'ERR_MOCHA_INVALID_REPORTER',)

Never got it working. It's working in vm. So seems that's the only workaround for me now.

Same issue for me, node 16.18.0. MacOS. Works fine same repo for Ubuntu 20 same node version.

Am testing in Node 16 locally on a Mac and it's fine. This may have self-resolved. Please ping if anyone continues to run into it.


Adding Dark Mode

The new menu site should support dark mode via the prefers-color-scheme media query like the main site does and the previous version did. Preferably using the same colors as previous iterations.

Added!


Wrappers for standard datasets

Using any dataset requires some initial mucking about to parse whatever custom formats the raw data and metadata (e.g. truth files) are in and get it into a suitable shape for processing. It would be great if this library (or an associated one) had a set of scripts/tools for working with a few of the most popular datasets.

I'm happy to help with this, but don't have that much spare time at the moment, so I'll do the cross validation issue first and see how it goes.

Thanks for opening this!
To be sure I totally understand - you mean that it would be nice to more easily use some common data sets. If so there are a few ways I think we can handle this:

Some data sets within this repository - and a small crate to load them (separate from rusty-machine).
Use a separate crate to handle this entirely - I know of this for example.
Place within rusty-machine behind a feature gate.

For me I think the second is the probably the best - though I'm not sure which data structure we'd use to store the data.
Just in case you're talking about some more general data handling - I tried my hand at it a while ago without much success. It's pretty tricky and I don't want something subpar inside rusty-machine. I think for now it's best to let users handle the more general data side on their own.
I'll keep this issue open as I think the idea is valuable. Making rusty-machine more accessible is good in my books and this helps with that for sure.

I wasn't aware of the library you linked in 2, thanks. That's the sort of thing I had in mind. I imagined something presenting a database-y view of locally downloaded datasets. There'd be a list of dataset metadata (including where the data's available from), and a local dataset repository supporting adding (downloading and converting to a standard format in a local directory) datasets from the list, removing them (deleting local caches), and loading the data (both the raw inputs, and e.g. ground truths) into a standard form. So that the user can acquire a dataset, load it and test some models, in a few lines.

I'm +1 to option 3 ATM.


[Android Nightly] $script rule doesn't work

.*.com/show.html?$script
Website: http://www.alleng.ru/edu/chem9.htm”
Request example: http://to.dedicatedgroups.com/show.html?GK3ozNSthJS4jenN-cnV-YSMoCSgzFzUiIS4_enc

Seems like it was caused by wrong logic of reapplying filters.
Related to #329


Objects with multiple identical xindexs on a dim aren't alignable

What happened?
I'm less up to speed on the xindex functionality, but a colleagues shared a similar issue to this, so some people are finding their way to them (good news!)
But one issue — it seems like here if we have an index u and v on the u dimension, we can't do something like foo + foo
What did you expect to happen?
Either it works, or — like the old days — it doesn't work but then we shouldn't allow getting the object into that state
Minimal Complete Verifiable Example
ds = xr.Dataset(
    data_vars=dict(
        x=(("u",), [1, 2, 3, 4]),
        y=(("u",), [100, 200, 300, 400]),
    ),
    coords=dict(
        u=[1, 2, 3, 4],
        v=(("u",), [1, 2, 1, 2]),
    ),
).set_xindex("v")

# <xarray.Dataset> Size: 128B
# Dimensions:  (u: 4)
# Coordinates:
#   * u        (u) int64 32B 1 2 3 4
#   * v        (u) int64 32B 1 2 1 2
# Data variables:
#     x        (u) int64 32B 1 2 3 4
#     y        (u) int64 32B 100 200 300 400

ds.x * ds.y

# Error below

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[X] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.
[X] Recent environment — the issue occurs with the latest version of xarray and its dependencies.

Relevant log output
File ~/workspace/xarray/xarray/core/_typed_ops.py:253, in DataArrayOpsMixin.__mul__(self, other)
    252 def __mul__(self, other: DaCompatible) -> Self:
--> 253     return self._binary_op(other, operator.mul)

File ~/workspace/xarray/xarray/core/dataarray.py:4747, in DataArray._binary_op(self, other, f, reflexive)
   4745 if isinstance(other, DataArray):
   4746     align_type = OPTIONS["arithmetic_join"]
-> 4747     self, other = align(self, other, join=align_type, copy=False)
   4748 other_variable_or_arraylike: DaCompatible = getattr(other, "variable", other)
   4749 other_coords = getattr(other, "coords", None)

File ~/workspace/xarray/xarray/core/alignment.py:887, in align(join, copy, indexes, exclude, fill_value, *objects)
    691 """
    692 Given any number of Dataset and/or DataArray objects, returns new
    693 objects with aligned indexes and dimension sizes.
   (...)
    877
    878 """
    879 aligner = Aligner(
    880     objects,
    881     join=join,
   (...)
    885     fill_value=fill_value,
    886 )
--> 887 aligner.align()
    888 return aligner.results

File ~/workspace/xarray/xarray/core/alignment.py:578, in Aligner.align(self)
    576 self.find_matching_indexes()
    577 self.find_matching_unindexed_dims()
--> 578 self.assert_no_index_conflict()
    579 self.align_indexes()
    580 self.assert_unindexed_dim_sizes_equal()

File ~/workspace/xarray/xarray/core/alignment.py:322, in Aligner.assert_no_index_conflict(self)
    318 pprint(self.indexes)
    319 items_msg = ", ".join(
    320     f"{k!r} ({v} conflicting indexes)" for k, v in dup.items()
    321 )
--> 322 raise ValueError(
    323     "cannot re-index or align objects with conflicting indexes found for "
    324     f"the following {msg}: {items_msg}\n"
    325     "Conflicting indexes may occur when\n"
    326     "- they relate to different sets of coordinate and/or dimension names\n"
    327     "- they don't have the same type\n"
    328     "- they may be used to reindex data along common dimensions"
    329 )

ValueError: cannot re-index or align objects with conflicting indexes found for the following dimensions: 'u' (2 conflicting indexes)
Conflicting indexes may occur when
- they relate to different sets of coordinate and/or dimension names
- they don't have the same type
- they may be used to reindex data along common dimensions

Anything else we need to know?


Support for multiple AI Search index calls / Documentation support for Semantic Kernel orchestration strategy

Motivation
This is rather a question than a feature request.
Is the code able to handle calls to multiple AI Search indexes (in the same or in different AI Search services)?
As I see, the CWYD accelerator allows for semantic kernel orchestration strategy.
Based on some readings,semantic kernel SDK includes the concept of a “planner“. The planner orchestrates multiple calls the Azure OpenAI service to make a plan to answer the user’s questions.
It also lets you create “plugins“, which can be calls out to native code.
Question is: is the code able to handle the said "plugins"?
Where and how to change it in order to include calls to multiple AI Search indexes?
How would you feel if this feature request was implemented?
Very happy!

Hey, any update ?

Hi @strudel0209 Thanks for reaching out. We have forwarded this request to our engineering team to add to our backlog. Will keep you posted.

any update?


Fix version check

Closes #1626
Description
Fixes regression in version checking

For contributor use:

[x] Targeted PR against master branch
[x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work
[x] Code follows the standards mentioned here
[x] Updated relevant documentation
[x] Re-reviewed Files changed in the Github PR explorer
[x] Added appropriate labels

Codecov Report
Attention: Patch coverage is 0% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 39.08%. Comparing base (464ec4a) to head (7b97ee4).

Files
Patch %
Lines

internal/command/command.go
0.00%
3 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1628      +/-   ##
==========================================
- Coverage   39.09%   39.08%   -0.02%
==========================================
  Files          52       52
  Lines        3535     3536       +1
==========================================
  Hits         1382     1382
- Misses       1978     1979       +1
  Partials      175      175

Flag
Coverage Δ

unittests
39.08% <0.00%> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


build.yaml specifying path of files to run moor_generator on

i'm seeing moor_generator:moor_generator on lib/bottomSheets/... .dart  (one of my widgets)
how can i limit moor_generator to only run lib\db ? i tried to see about this on https://moor.simonbinder.eu/docs/advanced-features/builder_options/
i saw one place online about build.yaml and a:
targets:
$default:
sources:
include: ["**"]
but, it's not clear to me how to specify path of files to run moor_generator on. thanks,

If you're running moor_generator in your default target (you probably are), you can limit the inputs of moor_generator like this:
targets:
  $default:
    builders:
      moor_generator|moor_generator:
        generate_for: ["lib/db/**"]

Setting the sources of the default target would be another option, but that affects other builders (like json_serializable or built_value) as well.
Also, feel free to ask questions like this in the discussions :)

If you're running moor_generator in your default target (you probably are), you can limit the inputs of moor_generator like this:
targets:
  $default:
    builders:
      moor_generator|moor_generator:
        generate_for: ["lib/db/**"]

Setting the sources of the default target would be another option, but that affects other builders (like json_serializable or built_value) as well.
Also, feel free to ask questions like this in the discussions :)


Remove Profiles from Open-iAPS Oref and move to Swift

Copied from @dnzxy's comment in Discord (https://discord.com/channels/1020905149037813862/1226878954535649351/1228787378529894411)
the way this currently works is this:

you create a profile or load a profile preset, both are stored in CoreData

that is then read into a Swift object in memory, and stored as another CoreData entity, Oref2_Variables

that is read again, then passed into JSON and passed to the oref algorithm

there all those variables, mostly boolean ones, are read from JSON, re-instantiated in the JS algo code and simply just to do some if else checks or to set a few variables to true/false or set %s

The this should work is all swift based and the only things that need to be passed to oref would be the current basal rate, the current isf rate, the current cr rate and all those modified by the selected %.
If you set the profile to disable SMBs, that can also happen Swift-based
determine basal should ideally only do exactly that: determine required insulin, not have a logic for adjusting setting metrics or basal rates.

Before refactoring code, need to be agree about the algorithm : See #214

Before refactoring code, need to be agree about the algorithm : See #214

What do you mean by „agree on algorithm“? 😊
Having a common understanding of how it currently works or agreeing on necessary changes? 🤔

What do you mean by „agree on algorithm“? 😊 Having a common understanding of how it currently works or agreeing on necessary changes? 🤔

Have a common understanding of how it currently works... For example TDD is based on 2 last hours and not 24 previous hours...
Secondly, what's the purpose of this enhancement ? Come back to the initial oref0 code or only remove override ? iAPS 4 has chosen to use the "initial" oref0 and to push all code in a specific javascript (prepare.js) called before the determine basal function - Principe of a middleware or proxy -. Do we go in the same target with Trio ? Do you move only in a swift code before calling the "initial" oref0 ?

Have a common understanding of how it currently works... For example TDD is based on 2 last hours and not 24 previous hours...

I’ll have to go in and look closer at this later, but I thought the 2 hours value was the 24hr TDD from each loop cycle from the past two hours averaged together.

Awesome, thanks for clarifying 😊 What iAPS has done is what most of the people involved here have said needs to be done; they were just quicker with it now, much to the quality of how it‘s been done if I’m frank.
Whatever we end up doing, please not do what they have done and pull Javascript out of Javascript only to put Javascript wrapped in JS worker into Swift, and duplicate data by storing them both in CoreData entities and JSON files.
I’d still vote for looking to utilize plain oref0 and additional layers should either adjust input or output; it does not belong into determine basal 😊 ideally, we‘d use what Milos / AAPS has rewritten in Kotlin 👍

Have a common understanding of how it currently works... For example TDD is based on 2 last hours and not 24 previous hours...

I’ll have to go in and look closer at this later, but I thought the 2 hours value was the 24hr TDD from each loop cycle from the past two hours averaged together.

That’s correct. A new 24 hour TDD value is calculated every five minutes. To reduce the impact of timing of meals (and associated meal bolus), the 24 TDD values over the last two hours is averaged. They still represent an estimate for a 24 hour TDD, but a smoother one than the raw five minute values.


update to INC2.0, refactor dir structure

Type of Change
INC 2.0 API update
Description
Updated examples/notebook/pytorch/ to INC2.0 API. Refactored directory structure
Expected Behavior & Potential Risk
Sample should run fine after proper env creation
How has this PR been tested?
PR has been tested on devcloud clx,icx,spr nodes
Dependency Change?
updated env creation, refer file scripts/devcloud_setup_env.sh

@NeoZhangJianyu Please review


Removing serde_derive in favor of serde feature derive.

Removes the need for extern crate

Favoring minserde, could use this in a feature config later.


Fix Telegram channel title in forwards

Forward from channels requires different handling than forward from the regular users.
This patch fixes the issue: it prints channel title instead of "forwarded from unknown".

Thanks!


add MASTER_PORT

add MASTER_PORT for redis server, default 6379
change DEFAULT_PORT to sentinel port, default 26379

@kadashu I'm really sorry for my late reply. Unfortunately, we cannot change DEFAULT_PORT since somebody can be already using it.
I think what we can do instead is introduce two new variable REDIS_PORT and SENTINEL_PORT with changes you provided. Regarding braking changes, we will need to set DEFAULT_PORT to value of REDIS_PORT afterwards.
What you think?

Hi @kadashu,
I reworked your PR and just committed it into master - s7anley/redis-sentinel-docker@deac3496f3fcd54c08b30cf5867164ba2792c2ed.


docs: added missing dependency to development installations

Added missing @nx/vite dependency to dev. installs.
PR Checklist
Please check if your PR fulfills the following requirements:

[x ] The commit message follows our guidelines: https://github.com/analogjs/analog/blob/main/CONTRIBUTING.md#-commit-message-guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ x] Documentation content changes
[ ] Other... Please describe:

Which package are you modifying?

[ ] vite-plugin-angular
[ ] vite-plugin-nitro
[ ] astro-angular
[ ] create-analog
[ ] router
[ ] platform
[ ] content
[ ] nx-plugin
[ ] trpc

What is the current behavior?
Incomplete documentation for installation of vitest into extant NG 17 project.
Closes #
What is the new behavior?
Following the docs leads to successful installation of vitest.
Does this PR introduce a breaking change?

[ ] Yes
[ x] No

Other information
None.

Thanks! @allcontributors add @iancharlesdouglas for doc


no difference between headerButtonLeft or headerButtonRight

I am trying to add a button to the navbar.
I am using the code in the template:
{{#contentFor "headerButtonRight"}}

{{/contentFor}}

and the button still shows on the left hand side
when I try
{{#contentFor "headerButtonLeft"}}

{{/contentFor}}

the button is still on the left hand side.

I had this issue too just now. I had to give my  tag the button class and then it moved back to the right side

I got it to work with slightly different syntax.
Here is what worked:
{{#contentFor "headerButtonRight"}}
    <button class="button button-clear" data-ion-modal="_contactsNew">{{> ionIcon icon="ios-loop-strong"}}</button>
  {{/contentFor}}

@sferoze glad you got it! i think it was just the button class that was important. I needed to use pathFor on mine which is why I used the anchor tag.


bootloader: Identify prep partition on loop device

While writing a bootc image to a qcow2 file, required partitions are
created using a loop device. On ppc64le arch the type value used
for prep boot partition is 41.
"partitions": [
{
"node": "/dev/loop0p1",
"start": 2048,
"size": 8192,
"type": "41",
"bootable": true
},{
"node": "/dev/loop0p2",
Expand the PReP Boot partition check logic to ensure bootloader
specific files are correctly written when a loop device is used.

Writing to a qcow2 image the bootloader specific information is correctly written to first partition (/dev/loop0p1)
Deploying container image...Freed objects: 118 bytes
done
Running bootupctl to install bootloader

bootupctl backend install --write-uuid --device /dev/loop0p1 /run/osbuild/mounts
Installed: grub.cfg
Trimming mounts

While writing a bootc image to a qcow2 file

You mean specifically with bootc-image-builder, correct? I think the real bug is https://github.com/osbuild/bootc-image-builder/pull/497#discussion_r1681487157

While writing a bootc image to a qcow2 file

You mean specifically with bootc-image-builder, correct? I think the real bug is osbuild/bootc-image-builder#497 (comment)

Yes, using bootc-image-builder.

(And the GPT type UUID should match)

Both dos and gpt are supported types. So yes, this can be changed to GPT (and the partition Type values 41 and 83 to 9E1A2D38-C612-4316-AA26-8B49521E5A8B & 0FC63DAF-8483-4772-8E79-3D69D8477DE4 respectively).
Let me know which is preferred?

Have updated the bootc-image-builder PR to use gpt format instead of dos. Closing this PR.


Trigger a hover event when mousing over dropzone (click to upload)

Could also just add a class.

Released in 0.8.1


Changes for ML-281

Problem: Currently Random Forest doesn't throw error if the maximum number of features to sample is greater than the total number of features.
Solution: Added ASSERT functions to throw an error when the situation occurs.

@johnholt Please review

I have updated the code according to @johnholt 's recommendations.

Looks good to me.  Approved.

@richardkchapman  Could you please merge


AttributeError Error 'NoneType' object has no attribute 'get'

https://counterparty.sentry.io/issues/6037375322/?project=4507062633365504&query=is%3Aunresolved issue.priority%3A[high%2C medium]&referrer=issue-stream&statsPeriod=7d&stream_index=0

already fixed here https://github.com/CounterpartyXCP/counterparty-core/commit/e1f4b71a8d6c30c9e8399e1be943ff2fbf1d91b6


feat: Add remote image caching behaviour docs

Description (required)

Currently the behaviour of Astro's remote image cache is undocumented, which makes discovering the feature difficult and its behaviour unclear. This change adds an explanation of the asset cache, as well as some technical details on how it functions, which should be helpful for anyone configuring a remote image server for Astro.
Part of this change references the revalidation feature introduced in Astro PR #12426, however the rest documents existing functionality.
Related issues & labels (optional)

Closes #
Suggested label: add new content

For Astro version: 5.1. See astro PR #12426.
Not yet merged - milestone.

Lunaria Status Overview
🌕 This pull request will trigger status changes.

Learn more
By default, every PR changing files present in the Lunaria configuration's files property will be considered and trigger status changes accordingly.
You can change this by adding one of the keywords present in the ignoreKeywords property in your Lunaria configuration file in the PR's title (ignoring all files) or by including a tracker directive in the merged commit's description.

Tracked Files

File
Note

en/guides/images.mdx
Source changed, localizations will be marked as outdated.

Warnings reference

Icon
Description

🔄
The source for this localization has been updated since the creation of this pull request, make sure all changes in the source have been applied.

Hello! Thank you for opening your first PR to Astro’s Docs! 🎉
Here’s what will happen next:

Our GitHub bots will run to check your changes.
If they spot any broken links you will see some error messages on this PR.
Don’t hesitate to ask any questions if you’re not sure what these mean!

In a few minutes, you’ll be able to see a preview of your changes on Netlify 🥳.

One or more of our maintainers will take a look and may ask you to make changes.
We try to be responsive, but don’t worry if this takes a few days.

It might be interesting to mention where the images are cached, sometimes users want to know what to need to keep between builds for caching (here you can link to cacheDir in the config)

I've moved the new documentation to its own section "Asset Caching", and mentioned how local images are also cached.
I added a mention of cacheDir in the body text instead of just the readmore link -
Is it okay to write the default cacheDir value here, or should I avoid repeating it and have people click through to the configuration reference?
I also expanded on revalidation, explaining that the remote server requires support for the If-Modified-Since and If-None-Match headers for revalidation to work. While this is a bit technical, if someone is having difficulty getting revalidation working it is probably best for this to be clearly documented.

Thanks @sarah11918 for the suggestions, I've committed them as-is.


Can the Reverse frequency function be added to MAKEFILE

Reverse frequency (F+8) is very practical, switching between direct frequencies
I have raised a lot of questions recently, and there are many things I don't understand. I'm very sorry.

I have discovered another issue. When you press the menu button, up and down buttons, etc., the nearby radio will receive a brief signal prompt.

Issues get totally lost if you mix them in other threads, you'll have to make a new issue for each issue.
F+8 should work ?

F+8 in 78da1d5 seems to work well. However (let's see if I can explain this), if you set a frequency in VFO with a TX Offset and move the frequency up or down, the offset symbol will disappear from the screen, and PTTing will not occur with displacement. That is, effectively the offset is disabled. BUT if you go to menu, you will see how the menu retains the offset (showing +/- instead of OFF), and if you press F+8 it will work like if the displacement is enable (and will reverse the frequency). Hope I explained well. I could record a video if I didn't.


Don't Merge! PR real time test for source branch full build

OPS Build status updates of commit 8b745c8:
:clock10: Full build: the average full build time is 6 min(s) 16 sec(s) This is based on the last 104 build(s) for this repository
:warning: This is a FULL build due to a configuration change (usually in openpublishing.publish.config.json and/or docfx.json). It is strongly suggested to merge this configuration change after the business hours, as it could impact the overall build performance of other PRs on the '.openpublishing.publish.config.json' branch. For more detailed info, please click here.

OPS Build status updates of commit 8b745c8:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

OPS Build status updates of commit 8b745c8:
:clock10: Full build: the average full build time is 6 min(s) 16 sec(s) This is based on the last 105 build(s) for this repository
:warning: This is a FULL build due to a configuration change (usually in openpublishing.publish.config.json and/or docfx.json). It is strongly suggested to merge this configuration change after the business hours, as it could impact the overall build performance of other PRs on the '.openpublishing.publish.config.json' branch. For more detailed info, please click here.

OPS Build status updates of commit 8b745c8:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details
Dependent Files

docfx.json
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Remove ndk-build

Most of the work for the CMake Android build has already been done.
Mainly it just needs work on the VulkanTests side. Which we largely already addressed with VVL. So it shouldn't be too hard.
FYI @lunarpapillo / @christophe-lunarg

Yes! :)


Added a cubic spline interpolator

Surprisingly, this approach turns out to be ~4x faster than
cs = scipy.interpolate.CubicSpline(x, y)
ys = cs(xs)

Though the comparison is really apples vs oranges, as the algorithm implemented is different.
There is loads of room for improvement (and potentially even a better algorithm - this is the best I could find in a short search), but this is good enough to be useful already.

Hmmm. Seems to be suffering from over-fitting. May be the algorithm itself... :disappointed:
array([  9.28148594e+04,   9.24955625e+04,   9.20495625e+04,
         9.14780625e+04,   9.07826562e+04,   8.99653906e+04,
         8.90285156e+04,   8.79746719e+04,   8.68069688e+04,
         8.55286875e+04,   8.41435469e+04,   8.26555156e+04,
         8.10689062e+04,   7.93884062e+04,   7.76193594e+04,
         7.57674844e+04,   7.38387656e+04,   7.18393594e+04,
         6.97743281e+04,   6.76511406e+04,   6.54768594e+04,
         6.32554688e+04,   6.09930781e+04,   5.86990938e+04,
         5.63856250e+04,   5.40647812e+04,   5.17401719e+04,
         4.94096406e+04,   4.70854688e+04,   4.47756719e+04,
         4.24794688e+04,   4.02053906e+04,   3.79599375e+04,
         3.57455156e+04,   3.35653594e+04,   3.14246094e+04,
         2.93292969e+04,   2.72865312e+04,   2.53046094e+04,
         2.33863750e+04,   2.15348125e+04,   1.97550469e+04,
         1.80409531e+04,   1.63841406e+04,   1.47842969e+04,
         1.32438281e+04,   1.17663438e+04,   1.03563906e+04,
         9.01753125e+03,   7.77323438e+03,   6.64531250e+03,
         5.62957812e+03,   4.71807812e+03,   3.90075000e+03,
         3.17331250e+03,   2.53498438e+03,   1.98378125e+03,
         1.51645312e+03,   1.12650000e+03,   8.08703125e+02,
         5.59656250e+02,   3.72718750e+02,   2.38671875e+02,
         1.46328125e+02,   8.44218750e+01,   4.56250000e+01,
         2.22500000e+01,   8.79687500e+00,   2.50000000e+00,
         5.15625000e-01])
array([ 301.09375 ,  300.53125 ,  300.015625,  299.4375  ,  298.765625,
        298.      ,  297.140625,  296.1875  ,  295.125   ,  293.953125,
        292.703125,  291.3125  ,  289.921875,  288.4375  ,  287.078125,
        285.5625  ,  284.078125,  282.453125,  280.578125,  279.15625 ,
        277.203125,  274.859375,  272.515625,  270.625   ,  269.5     ,
        268.96875 ,  266.875   ,  263.75    ,  262.6875  ,  259.71875 ,
        256.84375 ,  254.1875  ,  251.21875 ,  247.625   ,  243.890625,
        240.109375,  236.453125,  233.140625,  230.484375,  227.125   ,
        224.65625 ,  222.28125 ,  218.125   ,  213.25    ,  208.578125,
        203.90625 ,  199.796875,  195.921875,  192.28125 ,  195.171875,
        199.015625,  203.625   ,  207.703125,  210.421875,  214.25    ,
        218.203125,  223.078125,  227.640625,  229.859375,  232.703125,
        238.28125 ,  244.84375 ,  255.296875,  261.71875 ,  260.046875,
        271.171875,  243.546875,  208.515625,  170.84375 ,  223.03125 ])

%matplotlib notebook
import numpy as np
xs = np.linspace(0.1, 90000, 1000)[::-1]
print(press.data.shape)
ys_cubic = stratify.interpolate(xs, press.data[::3], temp.data[::3],
                          interpolation='cubic',
                          extrapolation='nan')

ys_linear = stratify.interpolate(xs, press.data[::3], temp.data[::3],
                          interpolation='linear',
                          extrapolation='nan')
plt.plot(ys_cubic, xs, label='cubic')
plt.plot(ys_linear, xs, label='linear')
plt.legend()
plt.gca().invert_yaxis()
plt.show()

I'm going to close this until future development is required...

Hey @pelson just noticed this.  Really cool, is this interest driven?  I see you closed it.
If you decide to get this back up, I'll happily take a look for you.

I didn't delete the branch for a reason 😄 .
Seems like an obvious extension to stratify, and one I'd love to see in here. Currently nobody actually needs this though (linear is typically good enough, even with atmospheric pressure, it seems)


Running error

Dear Dr.Hoohm,
I installed CITE-seq Count 1.4.3 ,
pip install CITE-seq-Count==1.4.3 It showed success installation under python3.7
but after I run the code:
CITE-seq-Count -R1 ####_R1_001.fastq.gz -R2 ####_001.fastq.gz -t samp1e_tga_list.csv -trim 22 -cbf 1 -cbl 16 -umif 17 -umil 26 -cells 1912 -wl ##_whitelist.csv -o ~/data/scRNAdata/test1/Citecount_test
It indicated errors like these:

/bin/sh: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)
Loading whitelist
Traceback (most recent call last):
File "/Software/CITE-Seq/bin/CITE-seq-Count", line 8, in
sys.exit(main())
File "/Software/CITE-Seq/lib/python3.7/site-packages/cite_seq_count/main.py", line 249, in main
ab_map = preprocessing.parse_tags_csv(args.tags)
File "/Software/CITE-Seq/lib/python3.7/site-packages/cite_seq_count/preprocessing.py", line 131, in parse_tags_csv
tags[row[0].strip()] = row[1].strip()
IndexError: list index out of range
What I should do to debug these? I have uploaded the white list and tag list files in case if your guys want to check,
Thank you,looking forward to hearing from you

G.zip

after I renew the format of tag.csv. the Error disappeared. Hope it will help others if anyone have the same error as mine


Add Shynet Template

Goal
Shynet is an analytics template for self-hosted website analytics and data visualization.
Payout
$150
Expectation
Contribute the Shynet template with an easy setup process for website tracking and data visualization. It should provide basic analytics, including unique visitors, popular pages, and referrers. Ensure that the template is lightweight and user-friendly. Clear documentation will assist users in effectively using Shynet for their analytics needs.

Hey Railway team ! Is it possible to take this task ?
Have a nice day ☀

Progression

[x] Setup deployments
[x] Admin user creation script
[x] Testing
[ ] Documentation of the template
[ ] Publishing of the template

Here is the repository with the template code : https://github.com/MatteoGauthier/shynet-railway

The template is now published with instructions ! Here is the template link ✅

Beautiful, will be reviewing this tomorrow my time.

Amazing work @MatteoGauthier , thank you for the submission.  I especially love how you included a railway.toml file, and your Overview/Readme is so good ✨
I'm happy to accept your submission and get the payout process going.  I have created a repository for you here: https://github.com/railwayapp-templates/shynet.  Please push your code to this repo and update the template configuration to point to this one.  Once done, please send me an email at<EMAIL_ADDRESS>and we'll finish it out.
Thanks again!

@melissa-hale I've just emailed you! Have a nice day, and thanks again for this opportunity


Updated add-password-reset-policy.md

Removed unnecessary explanation of what custom policies are
Linked custom policy starter pack

@bolt-io : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#label:"aq-pr-triaged"

@bolt-io : Thanks for your contribution! The author(s) have been notified to review your proposed change.


add tutorial how to run app in readme md

Will need a little more than this to merge. Add links to screenshots of the page. Take care of Capital letters. State the browser to open with etc.


C++ example: Restore the built-in PIN dialog

Resurrect the code that allows to show the PIN dialog as an HTML window
ordered by the App.
This PIN dialog is not recommended for general use, however it's still
useful in some scenarios - like requesting the PIN during the
"demo"/"test".

Thanks, done.


Enhancement: Sass?

Being trying to activate sass...put running into multiple errors? Had the same issues?

It's available. Please refer to the Vite documentation: https://github.com/vitejs/vite#css-pre-processors

@nexflo Use Sass styles just like in any other project utilizing Vue. E.g.:
import './styles.scss'

Or in a SFC:
<style lang="scss">
</style>


As a lobby leader I want to change the settings to change the number of rounds, the time per round and the topic of the words.

[ ] Only the lobby leader can change the settings.
[ ] As lobby leader you can choose a topic from a list of topics.
[ ] As lobby leader you can choose if the game is played with a chat or with a canvas or both.

97af585 af37bc6

af37bc6
97af585

af37bc6

af37bc69433486c7a063ea9c2f202a6194aaf8d7

97af58554f77f40fc51b8b33e09d0d59e2628c62

https://github.com/sopra-fs23-group-05/Client/commit/af37bc69433486c7a063ea9c2f202a6194aaf8d7


feat(exchanges): show no payment methods screen if no providers and exchanges are available

Description
Show the existing screen in app that says there are no payment options and invites them to switch their currency to the user if there are 0 payment methods.

Other changes
N/A
Tested
Added a test to SelectProvider.test.tsx to check for component visibility
CBPay card's visibility logic is moved to SelectProvider, and thus a majority of CoinbasePaymentSection tests have been migrated to SelectProvider.
How others should test
N/A
Related issues

Fixes #[299]
https://linear.app/valora/issue/ACT-299/only-show-crypto-exchanges-if-they-are-actually-available

Backwards compatibility
Brief explanation of why these changes are/are not backwards compatible.

would love to also see a screenshot of when there are providers, but none are exchanges (in which case there shouldn't be an exchanges section in the Select Providers screen)

would love to also see a screenshot of when there are providers, but none are exchanges (in which case there shouldn't be an exchanges section in the Select Providers screen)

Updated description in #2702.

Hey @MuckT  since NA is mentioned in how others should test section we are skipping this PR
CC: @awtsui
Thanks,!


Missing Class

[17:09:50] [Server thread/CRITICAL]: Error: "Class 'aieuo\mineflow\utils\Language' not found" (EXCEPTION) in "plugins/Mineflow-masterV2.phar/src/aieuo/mineflow/Main" at line 57

Where did you download it from?
The file may be corrupted, so please try to download it again.


Feature/update for chromium version and qpdf

Now chromium is from alpine 3.10 instead of edge, hopefully this fix package pinning issue. Alpine 3.10 doesn't have pdftk (which isn't maintained any more) so i replaced it with qpdf.
Edge repositories are still used for libgdiplus.
Edit: yeah build is broken, have to check that once at office.
Fixes #35

Closes #49


The color and clarity (pixels) of image resources, videos, text and other content are seriously distorted. . .

The color and clarity (pixels) of image resources, videos, text and other content are seriously distorted. . .
System environment:
Windows10 x64 19044
Chrome
Version 119.0.6045.200 (Official Build) (64-bit)
VSCode 1.85
To reproduce the example:

Open the same picture and the same video in Chrome and VSCode 1.85 respectively. However, it is surprising to find that the color and clarity (pixels) of image resources, videos, text and other content rendered by VSCode 1.85 are seriously distorted. . . The original image, video quality and colors look very distorted and terrible. . . Don't know what causes this?

Video 1 is rendered in the Chrome browser

Image 1 was rendered in the Chrome browser

Video 2 is rendered in VSCode 1.85 Markdown

Image 2 was rendered in VSCode 1.85 Markdown

You should be able to clearly see that the image quality, video quality and color rendered in VSCode 1.85 Markdown are very distorted. . . It is really difficult to understand that such color differences and color distortions will occur when using the same video, image, text and other content on the same computer. . .
Hopefully the VSCode team will fix and improve it in the next version. Thank you so much!

When images and videos are opened from the resource manager, the opened images and videos also have the above-mentioned color distortion and color difference problems. . .

Compare videos and images respectively:
Video 1 is rendered in the Chrome browser

Video 2 is rendered in VSCode 1.85 Markdown

Image 1 was rendered in the Chrome browser

Image 2 was rendered in VSCode 1.85 Markdown

When images and videos are opened from the resource manager, the opened images and videos also have the above-mentioned color distortion and color difference problems. . .

VSCode does not control the rendering of images and video, the support comes from the chromium runtime. If issue does not appear in Chrome then very likely issue is addressed in newer runtime versions. Closing as not actionable.


Connecting Blocks

Blocks currently cannot be connected via keyboard.
We need to figure out how to connect them as they are made as well as inserting them later.

Example to keep in mind: Need to be able to connect a block that is intended to be a value such as the number block and connect it backwards to a block it is compatible with.

Now able to add blocks to the screen and connect them using keystrokes. Still a bit buggy but will make issues as they arise.


Use parse instead of FromStr

This is more idiomatic.

Thank you for the fix! Merging.
This is a "Hello, world" application for Rust on Heroku, and it has been around since pre-Rust-1.0, so I'm very happy to accept pull requests to update anything that's getting stale.

Yeah I had my fork from long ago. :) this was still really useful to get something going today!
On Oct 1, 2016, 18:43 -0400, Eric Kidd<EMAIL_ADDRESS>wrote:

Merged #4 (https://github.com/emk/heroku-rust-cargo-hello/pull/4).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub (https://github.com/emk/heroku-rust-cargo-hello/pull/4#event-809449954), or mute the thread (https://github.com/notifications/unsubscribe-auth/AABsim-bQmRilu3u26kawpUp3eJSHC8Qks5qvuIVgaJpZM4KL4C6).


cfncluster configuration script crashed on EC2 instance.

Followed the
Platform: Ubuntu
Image Size: 8GB
Visibility: Public
17  $ sudo easy_install -U cfncluster
18  sudo easy_install -U cfncluster
19  cfncluster configure
AWS Region ID []: us-west-1
VPC Name [public]:
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/cfncluster", line 11, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python2.7/site-packages/cfncluster/cli.py", line 154, in main
    args.func(args)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/cfncluster/cli.py", line 25, in configure
    easyconfig.configure(args)
  File "/home/ubuntu/.local/lib/python2.7/site-packages/cfncluster/easyconfig.py", line 141, in configure
    key_name = prompt('Key Name', config.get('cluster ' + cluster_template, 'key_name') if config.has_option('cluster ' + cluster_template, 'key_name') else None, options=list_keys(aws_access_key_id, aws_secret_access_key, aws_region_name))
  File "/home/ubuntu/.local/lib/python2.7/site-packages/cfncluster/easyconfig.py", line 76, in list_keys
    keypairs = conn.get_all_key_pairs()
  File "/home/ubuntu/.local/lib/python2.7/site-packages/boto/ec2/connection.py", line 2836, in get_all_key_pairs
    [('item', KeyPair)], verb='POST')
  File "/home/ubuntu/.local/lib/python2.7/site-packages/boto/connection.py", line 1186, in get_list
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 401 Unauthorized
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error>AuthFailure<Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>f43a9c24-c4e9-487f-9b1d-90063d05fb3c</RequestID></Response>

Hello Zeev,
How did you resolve this issue? I am getting the same error.

I am getting this error also. What did you do to resolve this? Please post.

@PaulMcElligott I never resolved this issue.

I also have this issue...tried to user starcluster but doesn't work either...

The error message is that CfnCluster couldn't authenticate with the credentials provided.  The first prompts from CfnCluster should have been for an AWS Access Key and AWS Secret Key; are you sure you provided the correct ones?  It might be easier (and more secure) to skip hard coding those in the CfnCluster config script and instead follow the AWS CLI configuration instructions, so that you can ensure you have the correct credentials by first testing with the AWS CLI.  If you take the CLI credentials route, just hit enter when prompted for the Access and Secret Keys by CfnCluster.

My mistake was to try to develop my own AMI from StarCluster AMI that was on Ubuntu 13.04, which support has been withdrawn. That was my mistake. They have 3 or 4 StarCluster AMI that's still on that version.
I later used another StarCluster AMI that's on Ubuntu 14.04.4 (the only one that's later than 13.04), keyed-in all the correct AWS Access Key and AWS Secret Key and it's all OK!

Anyways, sorry to talk about starcluster. I also tried CfnCluster on that old AMI Ubuntu 13.04 and it also didn't work (before I realized the version issue).


Parallel Hessian tests fail locally

I have two tests failing locally, even on main:
test/test_parallel.py::test_hessian_bowl_np2[2]
test/test_parallel.py::test_hessian_bowl_np2[3]

The error messages are assertion errors for the metric against its expected value.

Yeah, just looking into this. This is all a bit weird: it appears that if in _compute_gradient_and_hessian you don't integrate by parts you get the right answer. I really don't understand why tbh, it looks all correct to me (IBP or not should be exactly the same in this case).

So I managed to produce the following MFE:
from firedrake import *

mesh = UnitSquareMesh(2, 2)
x, y = SpatialCoordinate(mesh)
V = VectorFunctionSpace(mesh, "CG", 1)
W = FunctionSpace(mesh, "CG", 1)
field = Function(W).interpolate(x)
g = TrialFunction(V)
phi = TestFunction(V)
sol = Function(V)
n = FacetNormal(mesh)

a = (
    inner(phi, g) * dx
)
L = (
    field * dot(phi, n) * ds
    - field * div(phi) * dx
)
solve(a == L, sol)
print(sol.dat.data)

which gives the expected result in serial (i.e. sol = (1,0)) but rubbish in parallel.
Could you sanity check the above and confirm you think this is a firedrake bug?

Thanks for looking into this @stephankramer. Yeah I can't see why your example would fail. The tests were passing locally for me previously and that part of the code hasn't changed so I would be inclined to think this cropped up last time I ran firedrake-update.

Hm so it works correctly in the firedrakeproject/firedrake:latest container (the MFE that is).

Oh yeah sorry I should mention that it passes fine on the CI, just fails for me locally.

The mfe and the tests pass locally for me as well (using the new docker image from https://github.com/pyroteus/animate/commit/b1a0acf4ce19ec899660662de5233f0330a1f79f) :)


Inappropriate parentheses removed

Before format:
test "panic op_get" {
  @deque.Deque::[1, 2, 3][-1] |> ignore
  (@deque.Deque::[1, 2, 3])[-1] |> ignore
}

then:
test "panic op_get" {
  @deque.Deque::[1, 2, 3][-1] |> ignore
  @deque.Deque::[1, 2, 3][-1] |> ignore
}

Based on the feedback I received on the coverage tool, the results obtained from these two writing methods are different, but the formatting tool removed parentheses for them.

Thank you for the report. This is an oversight, but we are deprecating the syntax T::[] so this shouldn't be an issue.


errors in console log

Hi,
We are running a POC with biggraphite considering it a replacement for our existing carbon/whisper setup.
We need to scale to ~4M datapoints per minute.
We are seeing truly massive numbers of errors in the console log of the form:
10/05/2017 13:27:20 :: Error writing to test4.agent.34.metrics.1843: None is not a Metric instance
10/05/2017 13:27:22 :: Unhandled Error
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/local/lib/python2.7/dist-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/opt/graphite/lib/carbon/writer.py", line 149, in writeForever
    writeCachedDataPoints()
--- <exception caught here> ---
  File "/opt/graphite/lib/carbon/writer.py", line 128, in writeCachedDataPoints
    state.database.write(metric, datapoints)
  File "<decorator-gen-1>", line 2, in write

  File "/usr/local/lib/python2.7/dist-packages/prometheus_client/core.py", line 879, in wrapped
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/biggraphite/plugins/carbon.py", line 117, in write
    self.accessor.insert_points_async(metric=metric, datapoints=datapoints)
  File "/usr/local/lib/python2.7/dist-packages/biggraphite/drivers/cassandra.py", line 1542, in insert_points_async
    metric, datapoints, on_done)
  File "/usr/local/lib/python2.7/dist-packages/biggraphite/accessor.py", line 823, in insert_points_async
    raise InvalidArgumentError("%s is not a Metric instance" % metric)
biggraphite.accessor.InvalidArgumentError: None is not a Metric instance

We are generating metrics using https://github.com/gorsuch/haggar on dedicated hosts
haggar forwards metrics via an elb to 3 biggraphite hosts which are currently running 2 instances of bg-carbon-cache loadbalanced by a carbon-c-relay instance. The Cassandra database cluster runs on the same hardware as bg-carbon-cache. The Biggraphite servers are c4.2xlarge instances (15GB memory, 8vcpu) with dedicated ssd backed volumes for cassandra.
So far we have managed to scale to ~400K datapoints per minute.
No real cassandra tuning has been done.
Suggestions / pointers  are much appreciated.

Hello ! Can you:

confirm which version / sha1 you are using
show your carbon.conf

I suspect that there is either something wrong with the metric cache on the carbon processes, or that you are using biggraphite+whisper in a configuration we never tried.
Note: for load-balancing multiple carbon instances, you might want to try this https://github.com/graphite-project/carbon/commit/a27561418a918560d81951dc7ea3886c9964bdb0 (and the following fix).

As discussed offline, this was certainly caused by the very low size of the metric cache.


Address SAM issue #1634 with ssc_auto_exec

The fundamental issue for the ssc_auto_exec in the EqnDatabase is that the EqnDatabase is constructed at load up on a page by page basis and has no idea on which UI variables will be available in whatever configuration the page is used. The ssc_auto_exec was constructed to avoid having to manually set each variable when the compute module referenced was called. Initially, any variables that were not to be set by the ssc_auto_exec function were designated by the UIHINT=”SIMULATION_PARAMETER” which is still in effect.
For SAM issue 1634, there is no way for ssc_auto_exec to have the same functionality as clicking the Simulate button in the UI. This is because ssc_auto_exec in constructed at page loadup when SAM starts and has no concept of a case or a configuration. That is, the list of UI variables is not available when the ssc_auto_exec function is evaluated.
To address SAM issue 1634 as much as possible, no UI variable will be used for any ssc variable that has the “REQUIRED_IF=*”. If the ssc variable is not required or has a default or a different REQUIRED_IF value, then ssc_auto_exec will skip the assignment to a UI variable.
Tested with add-temp-mspt_cmod_var branch of ssc - no equation errors

@tyneises, can you take a look at the failing MSLF configurations?

MSLF configurations fail because REQUIRED_IF is not set for all required variables.
add-temp-mspt_cmod_var and patch branch work fine for MSPT configurations.

Added fcall_var_exists to facilitate desired functionality of #1634
Working and running for add-temp-mspt_cmod_var and patch branches of ssc as requested.
@tyneises, @taylorbrown75 please review and let me know if you have any questions.

Logic is clear and works well. I produced the following table showing performance depending on required_if type and whether the variable is defined in the UI. Works as described.

Seems we can stop using SIMULATION_PARAMETER? Conditional required_if works with sim_type. (i.e. use REQUIRED_IF="sim_type=1")

@taylorbrown75 thanks for the detailed review!
@tyneises can we get stop using SIMULATION_PARAMETER as @taylorbrown75 suggests?

Updated table to include conditional required_if case.


Recipe for iHESP Global Datasets

Source Dataset

iHESP is focused on high-resolution, coupled climate simulations spanning the entire globe and regionally downscaled simulations of a region of interest (ex: Gulf of Mexico). Our global climate datasets have been generated using a high‐resolution configuration of the Community Earth System Model version 1.3 (CESM1.3), with a nominal horizontal resolution of 0.25° for the atmosphere and land models and 0.1° for the ocean and sea‐ice models. At these resolutions, the model permits tropical cyclones and ocean mesoscale eddies, allowing interactions between these synoptic and mesoscale phenomena with large‐scale circulations.

Link to the website / online documentation for the data:

https://ihesp.tamu.edu/products/ihesp-products/Datasets.html
https://ihesp.tamu.edu/products/ihesp-products/data-release/PI_control/index.html
https://ihesp.tamu.edu/products/ihesp-products/data-release/PI_control_phase2/index.html
https://ihesp.tamu.edu/products/ihesp-products/data-release/HIST_TNST/index.html
https://ihesp.tamu.edu/products/ihesp-products/data-release/RCP85_TNST/index.html

The file format (e.g. netCDF, csv): netCDF
How are the source files organized? (e.g. one file per day): monthly output
How are the source files accessed (e.g. FTP): THREDDS server with HTTP and OPeNDAP endpoints

https://datahub.geos.tamu.edu:8880/thredds/catalog/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/catalog.html
https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc

Any special steps required to access the data (e.g. password required): No

Transformation / Alignment / Merging

The data should merge cleanly.
Output Dataset

Zarr with all time slice files unified into a single timeseries. Possibly merging of different variables as well.
cc @paigem, @abishekg7

Reporting back from the OSM tutorial, where @selipot, Ian Carroll, and I tried our luck with this.
We unfortunately failed to even open datasets from within the binder environment. Using the example url
https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc, which I can download to my local computer by clicking it, but was unable to load into xarray:
path = 'https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc'
xr.open_dataset(path)

gives:
Note:Caching=1
Error:curl error: SSL peer certificate or SSH remote key was not OK
curl error details:
Warning:oc_open: Could not read url
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock)
    198             try:
--> 199                 file = self._cache[self._key]
    200             except KeyError:

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/lru_cache.py in __getitem__(self, key)
     52         with self._lock:
---> 53             value = self._cache[key]
     54             self._cache.move_to_end(key)

KeyError: [<class 'netCDF4._netCDF4.Dataset'>, ('https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc',), 'r', (('clobber', True), ('diskless', False), ('format', 'NETCDF4'), ('persist', False))]

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
/var/folders/_1/1k9jtjl51z333f21s7yht0340000gn/T/ipykernel_1917/1179327591.py in <module>
----> 1 xr.open_dataset(path)

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/api.py in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, backend_kwargs, *args, **kwargs)
    493
    494     overwrite_encoded_chunks = kwargs.pop("overwrite_encoded_chunks", None)
--> 495     backend_ds = backend.open_dataset(
    496         filename_or_obj,
    497         drop_variables=drop_variables,

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/netCDF4_.py in open_dataset(self, filename_or_obj, mask_and_scale, decode_times, concat_characters, decode_coords, drop_variables, use_cftime, decode_timedelta, group, mode, format, clobber, diskless, persist, lock, autoclose)
    548
    549         filename_or_obj = _normalize_path(filename_or_obj)
--> 550         store = NetCDF4DataStore.open(
    551             filename_or_obj,
    552             mode=mode,

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/netCDF4_.py in open(cls, filename, mode, format, group, clobber, diskless, persist, lock, lock_maker, autoclose)
    377             netCDF4.Dataset, filename, mode=mode, kwargs=kwargs
    378         )
--> 379         return cls(manager, group=group, mode=mode, lock=lock, autoclose=autoclose)
    380
    381     def _acquire(self, needs_lock=True):

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/netCDF4_.py in __init__(self, manager, group, mode, lock, autoclose)
    325         self._group = group
    326         self._mode = mode
--> 327         self.format = self.ds.data_model
    328         self._filename = self.ds.filepath()
    329         self.is_remote = is_remote_uri(self._filename)

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/netCDF4_.py in ds(self)
    386     @property
    387     def ds(self):
--> 388         return self._acquire()
    389
    390     def open_store_variable(self, name, var):

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/netCDF4_.py in _acquire(self, needs_lock)
    380
    381     def _acquire(self, needs_lock=True):
--> 382         with self._manager.acquire_context(needs_lock) as root:
    383             ds = _nc4_require_group(root, self._group, self._mode)
    384         return ds

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/contextlib.py in __enter__(self)
    117         del self.args, self.kwds, self.func
    118         try:
--> 119             return next(self.gen)
    120         except StopIteration:
    121             raise RuntimeError("generator didn't yield") from None

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/file_manager.py in acquire_context(self, needs_lock)
    185     def acquire_context(self, needs_lock=True):
    186         """Context manager for acquiring a file."""
--> 187         file, cached = self._acquire_with_cache_info(needs_lock)
    188         try:
    189             yield file

~/miniconda/envs/pangeo_forge_cm26/lib/python3.9/site-packages/xarray/backends/file_manager.py in _acquire_with_cache_info(self, needs_lock)
    203                     kwargs = kwargs.copy()
    204                     kwargs["mode"] = self._mode
--> 205                 file = self._opener(*self._args, **kwargs)
    206                 if self._mode == "w":
    207                     # ensure file doesn't get overriden when opened again

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4.Dataset.__init__()

src/netCDF4/_netCDF4.pyx in netCDF4._netCDF4._ensure_nc_success()

OSError: [Errno -68] NetCDF: I/O failure: b'https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc'

We tried to download an example file with curl, which worked on my local machine from the terminal, which only succeeded from my local machine.
We did not get any further, but I would love to understand what went wrong here. I guess it is not technically necessary to look at the data first, but I do not know much about it, and so exploring it first seemed like a good idea.

Is that an opendap link?

Abishek from the (formerly) iHESP project here. Thanks for looking into it!
I think we've had some issues opening datasets via OpenDAP on our THREDDS server previously. If this is a pre-requisite in order to do the data ingestion to the cloud, we can try to prioritize this issue during this week or next. It's possible that our OpenDAP links worked via Ferret or ncdump, but not with the python netCDF libraries. I'll need to revisit this and confirm.

That would be great @abishekg7.  Many thanks

Actually we would prefer to get netCDF files, not opendap. Is there a direct netcdf file download link from the thredds server?

I think a link like this would usually work with wget or curl.
Our thredds server was down last evening, but we have restarted it now. Since it's a little confusing, here are some basic instructions to download the netcdf files from a browser.

Go to our thredds server link for a specific variable  and click on any of the links

From the resulting page, click on the HTTPServer link to start the download. Or copy the link to use with wget.

I can also give you wget script to make this a little easier.

I was able to download the data no problem
 wget --no-check-certificate https://datahub.geos.tamu.edu:8880/thredds/fileServer/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc

and open it with xarray
xr.open_dataset('B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.SSH.032001-032912.nc'

<xarray.Dataset>
Dimensions:             (z_t: 62, z_t_150m: 15, z_w: 62, z_w_top: 62, z_w_bot: 62, nlat: 2400, nlon: 3600, time: 120, d2: 2)
Coordinates:
  * z_t                 (z_t) float32 500.0 1.5e+03 ... 5.625e+05 5.875e+05
  * z_t_150m            (z_t_150m) float32 500.0 1.5e+03 ... 1.35e+04 1.45e+04
  * z_w                 (z_w) float32 0.0 1e+03 2e+03 ... 5.5e+05 5.75e+05
  * z_w_top             (z_w_top) float32 0.0 1e+03 2e+03 ... 5.5e+05 5.75e+05
  * z_w_bot             (z_w_bot) float32 1e+03 2e+03 3e+03 ... 5.75e+05 6e+05
    ULONG               (nlat, nlon) float64 ...
    ULAT                (nlat, nlon) float64 ...
    TLONG               (nlat, nlon) float64 ...
    TLAT                (nlat, nlon) float64 ...
  * time                (time) object 0320-02-01 00:00:00 ... 0330-01-01 00:0...
Dimensions without coordinates: nlat, nlon, d2
Data variables: (12/51)
    dz                  (z_t) float32 1e+03 1e+03 1e+03 ... 2.5e+04 2.5e+04
    dzw                 (z_w) float32 500.0 1e+03 1e+03 ... 2.5e+04 2.5e+04
    KMT                 (nlat, nlon) float64 ...
    KMU                 (nlat, nlon) float64 ...
    REGION_MASK         (nlat, nlon) float64 ...

    SSH                 (time, nlat, nlon) float32 ...
Attributes: (12/13)
    title:           B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02
    history:         Sat Jul 24 01:43:33 2021: ncap2 -A -s time=udunits(time,...
    Conventions:     CF-1.0; http://www.cgd.ucar.edu/cms/eaton/netcdf/CF-curr...
    contents:        Diagnostic and Prognostic Variables
    source:          CCSM POP2, the CCSM Ocean Component
    revision:        $Id: tavg.F90 56176 2013-12-20 18:35:46Z<EMAIL_ADDRESS>$
    ...              ...
    start_time:      This dataset was created on 2019-11-06 at 09:58:40.0
    cell_methods:    cell_methods = time: mean ==> the variable values are av...
    nsteps_total:    68119571
    tavg_sum:        2678400.0
    tavg_sum_qflux:  2678400.0
    NCO:             netCDF Operators version 4.8.1 (Homepage = http://nco.sf/...

So it should be straightforward to ingest this data with a vanilla XarrayZarrRecipe.

Ok I put some work into this as a last action of the day.
First some thoughts about the whole dataset: Ideally we would like to have all variables in a gigantic zarr store I assume?
I see a major challenge there to harmonize the filepatterns across different variables. Upon closer inspection, many of the different variables have completely different output periods:
See the SSH variable:

these files are saved in 10 year intervals
For comparison the UVEL data:

These are saved yearly! So we need a separate file pattern for each variable!
I might also be missing something here. Maybe @abishekg7 has more insight.
But additionally, there are some inconsistencies in the naming even within single variables. See the earlier files of SSH:

There is an irregularity (also seen in filesize) going on here (coinciding with another naming scheme).
So maybe this is not as 'vanilla' as assumed, but might be a good boundary pusher for pangeo-forge.
Either way I decided to start a recipe with an easier target: Start with SSH and only use the later years, where the save period is homogenous.
So this is what I came up with:
from pangeo_forge_recipes.patterns import ConcatDim, FilePattern
from pangeo_forge_recipes.recipes import XarrayZarrRecipe
input_url_pattern = (
    "https://datahub.geos.tamu.edu:8880/thredds/fileServer/iHESPDataHUB"
    "/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH"
    "/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.nday1.SSH.{yyyystart}01-{yyyystop}12.nc"
)

def format_function(time):
    # I wonder if there is a way to name this simply year, and still have it concat along 'time'
    # I think that would be more intuitive here, but not a big deal.
    return input_url_pattern.format(
        yyyystart=f"{time:04d}",
        yyyystop=f"{time+9:04d}",
    )

# there is a break in the naming convention+ output timespan for some variables after year ~337.
# Lets only take years 340-510 for now
# years = range(340, 520, 10)
# for testing
years = range(340, 360, 10)

concat_dim = ConcatDim(name="time", keys=years, nitems_per_file=120)
pattern = FilePattern(format_function, concat_dim)

recipe = XarrayZarrRecipe(pattern, target_chunks={'time':6})
recipe

But when I try to test the caching I get a similar error to the one I got earlier:
for input_file in recipe.inputs_for_chunk(all_chunks[0]):
    recipe.cache_input(input_file)

pangeo_forge_recipes.recipes.xarray_zarr - INFO - Caching input 'Index({DimIndex(name='time', index=0, sequence_len=2, operation=<CombineOp.CONCAT: 2>)})'
pangeo_forge_recipes.storage - INFO - Caching file 'https://datahub.geos.tamu.edu:8880/thredds/fileServer/iHESPDataHUB/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.nday1.SSH.034001-034912.nc'
pangeo_forge_recipes.storage - INFO - Copying remote file 'https://datahub.geos.tamu.edu:8880/thredds/fileServer/iHESPDataHUB/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.nday1.SSH.034001-034912.nc' to cache
---------------------------------------------------------------------------
SSLCertVerificationError                  Traceback (most recent call last)
File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:986, in TCPConnector._wrap_create_connection(self, req, timeout, client_error, *args, **kwargs)
    985     async with ceil_timeout(timeout.sock_connect):
--> 986         return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
    987 except cert_errors as exc:

File /srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py:1081, in BaseEventLoop.create_connection(self, protocol_factory, host, port, ssl, family, proto, flags, sock, local_addr, server_hostname, ssl_handshake_timeout, happy_eyeballs_delay, interleave)
   1078         raise ValueError(
   1079             f'A Stream Socket was expected, got {sock!r}')
-> 1081 transport, protocol = await self._create_connection_transport(
   1082     sock, protocol_factory, ssl, server_hostname,
   1083     ssl_handshake_timeout=ssl_handshake_timeout)
   1084 if self._debug:
   1085     # Get the socket from the transport because SSL transport closes
   1086     # the old socket and creates a new SSL socket

File /srv/conda/envs/notebook/lib/python3.9/asyncio/base_events.py:1111, in BaseEventLoop._create_connection_transport(self, sock, protocol_factory, ssl, server_hostname, server_side, ssl_handshake_timeout)
   1110 try:
-> 1111     await waiter
   1112 except:

File /srv/conda/envs/notebook/lib/python3.9/asyncio/sslproto.py:528, in SSLProtocol.data_received(self, data)
    527 try:
--> 528     ssldata, appdata = self._sslpipe.feed_ssldata(data)
    529 except (SystemExit, KeyboardInterrupt):

File /srv/conda/envs/notebook/lib/python3.9/asyncio/sslproto.py:188, in _SSLPipe.feed_ssldata(self, data, only_handshake)
    186 if self._state == _DO_HANDSHAKE:
    187     # Call do_handshake() until it doesn't raise anymore.
--> 188     self._sslobj.do_handshake()
    189     self._state = _WRAPPED

File /srv/conda/envs/notebook/lib/python3.9/ssl.py:944, in SSLObject.do_handshake(self)
    943 """Start the SSL/TLS handshake."""
--> 944 self._sslobj.do_handshake()

SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

The above exception was the direct cause of the following exception:

ClientConnectorCertificateError           Traceback (most recent call last)
File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/implementations/http.py:383, in HTTPFileSystem._info(self, url, **kwargs)
    381 try:
    382     info.update(
--> 383         await _file_info(
    384             url,
    385             size_policy=policy,
    386             session=session,
    387             **self.kwargs,
    388             **kwargs,
    389         )
    390     )
    391     if info.get("size") is not None:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/implementations/http.py:732, in _file_info(url, session, size_policy, **kwargs)
    731 elif size_policy == "get":
--> 732     r = await session.get(url, allow_redirects=ar, **kwargs)
    733 else:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/client.py:535, in ClientSession._request(self, method, str_or_url, params, data, json, cookies, headers, skip_auto_headers, auth, allow_redirects, max_redirects, compress, chunked, expect100, raise_for_status, read_until_eof, proxy, proxy_auth, timeout, verify_ssl, fingerprint, ssl_context, ssl, proxy_headers, trace_request_ctx, read_bufsize)
    534         assert self._connector is not None
--> 535         conn = await self._connector.connect(
    536             req, traces=traces, timeout=real_timeout
    537         )
    538 except asyncio.TimeoutError as exc:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:542, in BaseConnector.connect(self, req, traces, timeout)
    541 try:
--> 542     proto = await self._create_connection(req, traces, timeout)
    543     if self._closed:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:907, in TCPConnector._create_connection(self, req, traces, timeout)
    906 else:
--> 907     _, proto = await self._create_direct_connection(req, traces, timeout)
    909 return proto

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:1206, in TCPConnector._create_direct_connection(self, req, traces, timeout, client_error)
   1205 assert last_exc is not None
-> 1206 raise last_exc

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:1175, in TCPConnector._create_direct_connection(self, req, traces, timeout, client_error)
   1174 try:
-> 1175     transp, proto = await self._wrap_create_connection(
   1176         self._factory,
   1177         host,
   1178         port,
   1179         timeout=timeout,
   1180         ssl=sslcontext,
   1181         family=hinfo["family"],
   1182         proto=hinfo["proto"],
   1183         flags=hinfo["flags"],
   1184         server_hostname=hinfo["hostname"] if sslcontext else None,
   1185         local_addr=self._local_addr,
   1186         req=req,
   1187         client_error=client_error,
   1188     )
   1189 except ClientConnectorError as exc:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/aiohttp/connector.py:988, in TCPConnector._wrap_create_connection(self, req, timeout, client_error, *args, **kwargs)
    987 except cert_errors as exc:
--> 988     raise ClientConnectorCertificateError(req.connection_key, exc) from exc
    989 except ssl_errors as exc:

ClientConnectorCertificateError: Cannot connect to host datahub.geos.tamu.edu:8880 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')]

The above exception was the direct cause of the following exception:

FileNotFoundError                         Traceback (most recent call last)
Input In [56], in <cell line: 1>()
      1 for input_file in recipe.inputs_for_chunk(all_chunks[0]):
----> 2     recipe.cache_input(input_file)

File ~/pangeo-forge-recipes/pangeo_forge_recipes/recipes/xarray_zarr.py:924, in XarrayZarrRecipe.cache_input(self, input_key)
    919 """Cache an input
    920
    921 :param chunk_key: Which input to cache
    922 """
    923 warnings.warn(_deprecation_message, DeprecationWarning)
--> 924 cache_input(input_key, config=self)

File ~/pangeo-forge-recipes/pangeo_forge_recipes/recipes/xarray_zarr.py:155, in cache_input(input_key, config)
    153     logger.info(f"Caching input '{input_key!s}'")
    154     fname = config.file_pattern[input_key]
--> 155     config.storage_config.cache.cache_file(
    156         fname,
    157         config.file_pattern.query_string_secrets,
    158         **config.file_pattern.fsspec_open_kwargs,
    159     )
    161 if config.cache_metadata:
    162     if config.storage_config.metadata is None:

File ~/pangeo-forge-recipes/pangeo_forge_recipes/storage.py:166, in CacheFSSpecTarget.cache_file(self, fname, secrets, **open_kwargs)
    164 target_opener = self.open(fname, mode="wb")
    165 logger.info(f"Copying remote file '{fname}' to cache")
--> 166 _copy_btw_filesystems(input_opener, target_opener)

File ~/pangeo-forge-recipes/pangeo_forge_recipes/storage.py:37, in _copy_btw_filesystems(input_opener, output_opener, BLOCK_SIZE)
     36 def _copy_btw_filesystems(input_opener, output_opener, BLOCK_SIZE=10_000_000):
---> 37     with input_opener as source:
     38         with output_opener as target:
     39             start = time.time()

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/core.py:103, in OpenFile.__enter__(self)
    100 def __enter__(self):
    101     mode = self.mode.replace("t", "").replace("b", "") + "b"
--> 103     f = self.fs.open(self.path, mode=mode)
    105     self.fobjects = [f]
    107     if self.compression is not None:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/spec.py:1030, in AbstractFileSystem.open(self, path, mode, block_size, cache_options, compression, **kwargs)
   1028 else:
   1029     ac = kwargs.pop("autocommit", not self._intrans)
-> 1030     f = self._open(
   1031         path,
   1032         mode=mode,
   1033         block_size=block_size,
   1034         autocommit=ac,
   1035         cache_options=cache_options,
   1036         **kwargs,
   1037     )
   1038     if compression is not None:
   1039         from fsspec.compression import compr

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/implementations/http.py:343, in HTTPFileSystem._open(self, path, mode, block_size, autocommit, cache_type, cache_options, size, **kwargs)
    341 kw["asynchronous"] = self.asynchronous
    342 kw.update(kwargs)
--> 343 size = size or self.info(path, **kwargs)["size"]
    344 session = sync(self.loop, self.set_session)
    345 if block_size and size:

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/asyn.py:91, in sync_wrapper.<locals>.wrapper(*args, **kwargs)
     88 @functools.wraps(func)
     89 def wrapper(*args, **kwargs):
     90     self = obj or args[0]
---> 91     return sync(self.loop, func, *args, **kwargs)

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/asyn.py:71, in sync(loop, func, timeout, *args, **kwargs)
     69     raise FSTimeoutError from return_result
     70 elif isinstance(return_result, BaseException):
---> 71     raise return_result
     72 else:
     73     return return_result

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/asyn.py:25, in _runner(event, coro, result, timeout)
     23     coro = asyncio.wait_for(coro, timeout=timeout)
     24 try:
---> 25     result[0] = await coro
     26 except Exception as ex:
     27     result[0] = ex

File /srv/conda/envs/notebook/lib/python3.9/site-packages/fsspec/implementations/http.py:396, in HTTPFileSystem._info(self, url, **kwargs)
    393     except Exception as exc:
    394         if policy == "get":
    395             # If get failed, then raise a FileNotFoundError
--> 396             raise FileNotFoundError(url) from exc
    397         logger.debug(str(exc))
    399 return {"name": url, "size": None, **info, "type": "file"}

FileNotFoundError: https://datahub.geos.tamu.edu:8880/thredds/fileServer/iHESPDataHUB/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.nday1.SSH.034001-034912.nc

Note that I can download the file manually using:
!wget --no-check-certificate https://datahub.geos.tamu.edu:8880/thredds/fileServer/iHESPDataHUB/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02/ocn/SSH/B.E.13.B1850C5.ne120_t12.sehires38.003.sunway_02.pop.h.nday1.SSH.034001-034912.nc

The key there is the --no-check-certificate flag. Is there a way to pass something similar to the recipe?
Happy to keep tinkering further but for today Ill clock out and watch some 🏀


How parachain collators get incentived in connection model?

Here is some introduction about collaters:
https://wiki.polkadot.network/en/latest/polkadot/learn/parachains/#collators

Parachains are operated and new blocks are authored by collator nodes that maintain "full-nodes" of a particular parachain. They retain all necessary information to be able to propose new blocks and execute transactions in a similar way as miners do in traditional PoW blockchains. A block proposed by a collator node will be checked by the validator node before it is accepted as a canonical state transition by the relay-chain.

Can the validators in Solo model continue contributing as collators (get reward at the same time) after darwinia switching to connection module?

They only get incentive for their up time, no consensus security contributions


Allow for publishing to our local maven repo.

@Wikia/services-team

LGTM

btw.

Are we still need this?
Local name in PR name is confusing. Internal sounds better


upgrade pipelines to CUDA 12.2

Purpose
This should hopefully fix the InitError: CUDA error (code 999, CUDA_ERROR_UNKNOWN) issues
To-do

Content

[ ] I have read and checked the items on the review checklist.

bors r+


Fix iwyu include path

This PR fixes the third-party library include paths for the iwyu (include-what-you-use) tool.

Hi @jianqiao
This looks good to me. Have you been actively using the IWYU library? I stopped using it a while ago because of the false positives. I am not sure if it has improved.

Merged.


Mass Compile G CLI Tools After Install

If we update the G CLI tool it can mean that the G CLI Tools installed have to recompile and if an API has changed it may cause a save dialog. This will cause problems with CI builds since the dialog will block.
We could consider mass-compiling this directory after this install to eliminate this problem

#80 should mean that the system doesn't lock up in this case, making this less urgent.