pmap inside jit causes assert not ragged assertion

Description
I am aware that it may not be a good idea to use pmap inside jit https://github.com/google/jax/issues/2926 but I want to use pmap inside the cost function passed to jax.numpy.linalg.cg() and which jit compiles it for me, so I have no choice...
Running the following minimal example on 16 Tesla V100 GPUs demonstrates the problem (here I jit manually):
import jax
from jax import numpy as np
from jax import lax

for batches in [
        [np.ones((2, 2)), np.ones((14, 1))], # works
        [np.ones((3, 2))],                   # works
        [np.ones((13, 1))],                  # works
        [np.ones((13, 1)), np.ones((3, 2))], # pmap fails
]:
    def compute_batch(batch):
        return np.ones((7))

    @jax.jit
    def pmap_over_batches(batches):
        ret = [
            jax.pmap(
                compute_batch,
                devices=jax.devices()[:len(batch)],
                )(batch).flatten()
            for batch in batches
        ]
        return ret

    pmap_ret = pmap_over_batches(batches)
    print("pmap done", [batch.shape for batch in batches], pmap_ret)

The third value in the for loop triggers a assert not ragged assertion
Traceback (most recent call last):
  File "test_pmap.py", line 39, in <module>
    pmap_ret = pmap_over_batches(batches)
  File "test_pmap.py", line 28, in pmap_over_batches
    ret = [
  File "test_pmap.py", line 29, in <listcomp>
    jax.pmap(
jax._src.source_info_util.JaxStackTraceBeforeTransformation: AssertionError

The preceding stack trace is the source of the JAX operation that, once transformed by JAX, triggered the following exception.

--------------------

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test_pmap.py", line 39, in <module>
    pmap_ret = pmap_over_batches(batches)
  File "/.../lib/python3.8/site-packages/jax/_src/traceback_util.py", line 162, in reraise_with_filtered_traceback
    return fun(*args, **kwargs)
  File "/.../lib/python3.8/site-packages/jax/_src/api.py", line 473, in cache_miss
    out_flat = xla.xla_call(
  File "/.../lib/python3.8/site-packages/jax/core.py", line 1767, in bind
    return call_bind(self, fun, *args, **params)
  File "/.../lib/python3.8/site-packages/jax/core.py", line 1783, in call_bind
    outs = top_trace.process_call(primitive, fun_, tracers, params)
  File "/.../lib/python3.8/site-packages/jax/core.py", line 680, in process_call
    return primitive.impl(f, *tracers, **params)
  File "/.../lib/python3.8/site-packages/jax/_src/dispatch.py", line 182, in _xla_call_impl
    compiled_fun = _xla_callable(fun, device, backend, name, donated_invars,
  File "/.../lib/python3.8/site-packages/jax/linear_util.py", line 285, in memoized_fun
    ans = call(fun, *args)
  File "/.../lib/python3.8/site-packages/jax/_src/dispatch.py", line 230, in _xla_callable_uncached
    return lower_xla_callable(fun, device, backend, name, donated_invars, False,
  File "/.../lib/python3.8/site-packages/jax/_src/profiler.py", line 206, in wrapper
    return func(*args, **kwargs)
  File "/.../lib/python3.8/site-packages/jax/_src/dispatch.py", line 340, in lower_xla_callable
    module, keepalive = mlir.lower_jaxpr_to_module(
  File "/.../lib/python3.8/site-packages/jax/interpreters/mlir.py", line 546, in lower_jaxpr_to_module
    lower_jaxpr_to_fun(
  File "/.../lib/python3.8/site-packages/jax/interpreters/mlir.py", line 800, in lower_jaxpr_to_fun
    out_vals, tokens_out = jaxpr_subcomp(ctx.replace(name_stack=callee_name_stack),
  File "/.../lib/python3.8/site-packages/jax/interpreters/mlir.py", line 918, in jaxpr_subcomp
    ans = rule(rule_ctx, *map(_unwrap_singleton_ir_values, in_nodes),
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1700, in _pmap_lowering
    outs = [_mhlo_unshard(aval, new_env, out_axis, shard,
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1700, in <listcomp>
    outs = [_mhlo_unshard(aval, new_env, out_axis, shard,
  File "/.../lib/python3.8/site-packages/jax/interpreters/pxla.py", line 1649, in _mhlo_unshard
    xla.axis_groups(axis_env, axis_env.names[-1]))
  File "/.../lib/python3.8/site-packages/jax/interpreters/xla.py", line 360, in axis_groups
    assert not ragged
jax._src.traceback_util.UnfilteredStackTrace: AssertionError

The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.

--------------------

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "test_pmap.py", line 39, in <module>
    pmap_ret = pmap_over_batches(batches)
AssertionError

very similar to the one reported here for scan https://github.com/google/jax/issues/2018 . Other batch shapes also trigger the error. Some work, some don't. I wasn't able to find any pattern.
The same error is triggered when spoofing 16 devices in a CPU only environment.
Because of the advice here https://github.com/google/jax/issues/13858 I have also tried v0.3.10 but got the same result as on v0.4.1.
What jax/jaxlib version are you using?
jax 0.4.1, jaxlib 0.4.1+cuda11.cudnn86
Which accelerator(s) are you using?
CPU, GPU
Additional system info
Python 3.8.12, Linux
NVIDIA GPU info
No response

Thanks for raising this, with the clear reproducer!
The issue is that we compute the number of replicas needed by the jitted computation with a max over all the numbers of replicas needed by each pmap inside it, but actually we need a least common multiple.
Here's another reproducer:
import jax
import jax.numpy as jnp

@jax.jit
def f():
  a = jax.pmap(lambda x: x)(jnp.arange(2))
  b = jax.pmap(lambda x: x)(jnp.arange(3))
  return a, b

f()

Actually, we don't really need a least-common-multiple here. We just need that for nested pmaps; having a max across separate pmaps is right. So the #13938 fix isn't ideal yet: it'll pass these tests, but require too many devices. For example, for the code in the OP to run would require lcm(3, 13) = 39 devices.
But the 'axis_groups' calculations we perform while lowering to HLO do currently require this least-common-multiple logic. So to require fewer devices, we'll need to adapt that logic to create unevenly sized axis groups. (That's basically what the assertion is checking will never happen.)

but I want to use pmap inside the cost function passed to jax.numpy.linalg.cg() and which jit compiles it for me, so I have no choice...

You meant jax.scipy.sparse.linalg.cg right?

Awesome! Thanks for looking into this so quickly! Yes I mean jax.scipy.sparse.linalg.cg.
I was under the impression that the number of devices must match the 0th axis of batch, so for the case [np.ones((13, 1)), np.ones((3, 2))] the frst pmap in the list comprehension would run on 13 devices and the second on 3.
Is that no longer true after the function has been jit compiled?
In the actual use case I have a function that needs to compute a number of things that are not a multiple of the number of devices with pmap inside the cost function passed to jax.scipy.sparse.linalg.cg. I want to do that by calling pmap twice. Once with a batch whose 0th axis is equal to the number of devices and then a second call to spread the computation of the remained over a subset of devices.

Is there any chance of getting a full fix for this, which also solves the problem of requiring an excessive amount of devices? Or maybe a workaround how pmap could be called from a jit compiled function that avoids the problem in the first place? Thanks very much for your time and effort!


API client's should remove APIOptionsFunc type alias

The type alias generated into API Client package, APIOptionsFunc should be removed because it makes it harder for utility functions defined outside of the API client's package to be compatible with the APIOptionsFunc type alias.
Type alias of a function doesn't really add any benefit other than documentation, since no additional methods will be added to the type alias.
func main() {
	var fn MyFunc

	fn = Impl
	fmt.Println(fn(&SomeType{}))

	SomeConfig(SomeSetup()...)
}

type MyFunc func(*SomeType) error

func Impl(t *SomeType) error {
	return nil
}

func SomeConfig(fns ...MyFunc) {

}

type SomeType struct{}

func SomeSetup() []func(*SomeType) error {

	return []func(*SomeType) error{}
}

cannot use SomeSetup() (type []func(*SomeType) error) as type []MyFunc in argument to SomeConfig

https://play.golang.org/p/754BI_LsS_6

type alise removed


[FEATURE] Curly brackets

It would be great to have curly brackets { }, used as \{ and \} implemented.
Right now the workaround is to put them in normal text but this does not allow using subscripts and superscripts.

In the context of GLMakie and the LaTeXStrings packages you ca use an empty unicode character of width zero as follows:
using GLMakie
using LaTeXStrings
aem = Char(0x200B)
scatter(1:10, 1:10; figure = (; fontsize = 32),
    axis = (; title= latexstring("an equation: {\$y_i\$}\${$aem}_{a=1}^N\$")))


checking empty string in examples #2783

fixed #2783

included and replaced by #2874 (fixes the warning, but doesn't modify the original value)


support to_timestamp with optional chrono formats

Which issue does this PR close?
Closes 5398.
Rationale for this change
Adding flexibility for parsing timestamp strings
What changes are included in this PR?
Code, tests and user guide updates.
Are these changes tested?
Tests were added and pass.
Are there any user-facing changes?
dataframe.to_timestamp(..) changed from to_timestamp(col("a")) to to_timestamp(vec![col("a")])

Thanks @alamb I would probably stick to PG standard as we declared. The PR implementation is really cool, and we may want to have a separate function for it? like from_chrono or something

I had a thought for the future to allow for setting the format based on either config or based on dialect. I was thinking that might be best left to after the function refactor (https://github.com/apache/arrow-datafusion/issues/8045) is done then have separate implementations if desired. I just didn't want to write a parser for the PG syntax as it's pretty detailed and just wanted to get something out there. Someone else noted the complexity of implementing the PG syntax
Do you have happen to have such a parser available?

Thanks @alamb I would probably stick to PG standard as we declared. The PR implementation is really cool, and we may want to have a separate function for it? like from_chrono or something

I had a thought for the future to allow for setting the format based on either config or based on dialect. I was thinking that might be best left to after the function refactor (#8045) is done then have separate implementations if desired. I just didn't want to write a parser for the PG syntax as it's pretty detailed and just wanted to get something out there. Someone else noted the complexity of implementing the PG syntax
Do you have happen to have such a parser available?

I'm not sure if its available, but we prob use your approach to create an adapter to PG formatting? 🤔
So you map PG formatting rules into chrono formatting rules

It's possible to directly map most of the pg patterns to chrono patterns
but not all. It's a pretty deep rabbit hole - pg supports Julian Day which
chrono doesn't. PG also supports Era (AD/BC) whereas chrono only supports
+- on year. Then there are escape characters of which backslash is allowed (but
now discouraged
https://www.postgresql.org/docs/16/sql-syntax-lexical.html#SQL-SYNTAX-CONSTANTS
= S<IP_ADDRESS>), you can define your own in unicode strings, and we would
possibly have to handle all of the c-style escape sequences in the format.
And heaven forbid if you have to parse or display any day/month text in
anything but English. Want day 1 in a week to be Sunday in Chrono (numerous
Muslim countries, Israel) ? Nope, you get day 0 - 6 instead (PG does 1-7
correctly).
Or we just keep it simple for now :)
On Thu, Jan 18, 2024 at 4:58 PM comphead @.***> wrote:

Thanks @alamb https://github.com/alamb I would probably stick to PG
standard as we declared. The PR implementation is really cool, and we may
want to have a separate function for it? like from_chrono or something
I had a thought for the future to allow for setting the format based on
either config or based on dialect. I was thinking that might be best left
to after the function refactor (#8045
https://github.com/apache/arrow-datafusion/issues/8045) is done then
have separate implementations if desired. I just didn't want to write a
parser for the PG syntax as it's pretty detailed and just wanted to get
something out there. Someone else noted
https://github.com/apache/arrow-datafusion/issues/5398#issuecomment-1820970911
the complexity of implementing the PG syntax
Do you have happen to have such a parser available?
I'm not sure if its available, but we prob use your approach to create an
adapter to PG formatting? 🤔
So you map PG formatting rules into chrono formatting rules
—
Reply to this email directly, view it on GitHub
https://github.com/apache/arrow-datafusion/pull/8886#issuecomment-1899288250,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABXHUU5XOZ6ME4SMDBNXJDYPGLI5AVCNFSM6AAAAABB5MLLU6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJZGI4DQMRVGA
.
You are receiving this because you were mentioned.Message ID:
@.***>

--
Bruce Ritchie
Principal Software Engineer
Veeva Systems
@.*** https://profiles.google.com/?hl=en&tab=mX
Customer Master Simplified. Data, Software, and Services All-in-one with
Veeva Network
http://www.veeva.com/customer-master/?utm_source=google&utm_medium=email signature&utm_campaign=Network Email Signature Link

I'll leave the final decision to @alamb although he already approved.
My point it would unexpected to users to have to_timestamp behaving differently comparing to PG. I prefer a separate method for from_chrono conversions, but if we decide to go with current approach we need to provide more documentation and examples for users in scalar_functions.md. Ideally even to have a separate doc page with examples

I took the liberty of merging up from main and running prettier to get CI clean.

... if we decide to go with current approach we need to provide more documentation and examples for users in scalar_functions.md. Ideally even to have a separate doc page with examples

I did add a fairly extensive example sql however unless a user knows where to look they won't find see it. Speaking from experience adding more extensive documentation for functions would be very helpful for someone starting up with datafusion (& rust). I'll be happy to update the docs with a brief example and a link to the example sql - perhaps that could be a pattern for all functions?

I'll be happy to update the docs with a brief example and a link to the example sql - perhaps that could be a pattern for all functions?

That would be amazing @Omega359  -- and really helpful. I have found that when there are good existing patterns subsequent contributors follow them, but getting the initial pattern set is the tough part.

Thanks again @Omega359 and @comphead

@Omega359 Do you know why to_timestamp has more than 2 arguments? What behaviour does this function follow?
In Postgres, there are two arguments
to_timestamp ( text, text ) → timestamp with time zone

Converts string to time stamp according to the given format. (See also to_timestamp(double precision) in [Table 9.33](https://www.postgresql.org/docs/current/functions-datetime.html#FUNCTIONS-DATETIME-TABLE).)

to_timestamp('05 Dec 2000', 'DD Mon YYYY') → 2000-12-05 00:00:00-05

to_timestamp ( double precision ) → timestamp with time zone

Convert Unix epoch (seconds since 1970-01-01 00:00:00+00) to timestamp with time zone

to_timestamp(1284352323) → 2010-09-13 04:32:03+00

Can I change the signature to be consistent with Postgres one?
https://github.com/apache/datafusion/blob/7ebd993fe47e78006bad33256160716f73b13cef/datafusion/sqllogictest/test_files/timestamps.slt#L362-L366

Tracked in https://github.com/apache/datafusion/issues/13351

https://datafusion.apache.org/user-guide/sql/scalar_functions.html#to-timestamp
I would be very much opposed to changing this to be exactly like pg.


Why xformer can not load, and gpu usage is lower than cpu usage？

GPU 3060，cpu 12490F

My GPU load is 90% but on average 50-60%

I was running into the same issue and decided to make a virtual environment, open a terminal at the folder (i open the folder in vscode and then open a terminal at the folder path)
pip install virtualenv
virtualevn venv
.\venv\Scripts\activate
pip install -r requirements.txt
Got this error ERROR: Invalid requirement: 'torch=2.0.0' (from line 12 of requirements.txt)
Changed torch=2.0.0 to torch==2.0.0 in the requirements.txt file,  Make sure to save ctrl+s
To fix running it on CPU vs my 3090 I did pip install torch==2.0.0+cu118 torchvision==0.15.1+cu118 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118

When looking at the Task Manager, you need to click on where it says 3D and change it to CUDA to see the usage


Removing a bit of redundant syntax

cc @parasharrajat @stitesExpensify
Details
As mentioned in this comment: https://github.com/Expensify/App/pull/9865#discussion_r920237800
Fixed Issues
https://github.com/Expensify/App/pull/9865#discussion_r920237800

🚀 Deployed to production by @luacmartins in version: 1.1.85-8 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Add encoding to New-Migration

I added the possibility to add encoding to the "New-Migration" command.
My motivation was that we have (unfortunately) umlauts in some of our scripts and with the command the file is ANSI-encoded, so the umlauts got changed to some kind of question mark which caused trouble in our database.
If no or no valid encoding is passed it'll use the ANSI-encoding for the new migration file.

@bradyholt Would be happy if you could have a look at this

Thanks for the PR!

Released: https://www.nuget.org/packages/dbup-consolescripts/1.2.1


[release/v1.54] Update kubernetes-cni to v1.2.0 and cri-tools to v1.26.0

What this PR does / why we need it:
This is a manual backport of #1535 and #1534 to the release/v1.54 branch. This is needed for KubeOne 1.5 and the latest (January 2023) Kubernetes patch releases.
What type of PR is this?
/kind feature
Does this PR introduce a user-facing change? Then add your Release Note here:
Update kubernetes-cni to v1.2.0 and cri-tools to v1.26.0

Documentation:
NONE

/test pull-machine-controller-e2e-aws-spot-instance

/lgtm

/override pull-machine-controller-e2e-kubevirt
/override pull-machine-controller-e2e-vmware-cloud-director
/override pull-machine-controller-e2e-vsphere
/override pull-machine-controller-e2e-aws-spot-instance


Upgrade to gliderlabs/alpine:3.3.

LGTM in running the tests. Unfortunately it looks like a slight bump in total image size. Also I noticed there is now a 3.4 so I'll go ahead and test that one later and re-release again.


Where is the data from?

I was reading through it and wondering where the data was from? Mechanical Turk?  Another website?

All the keywords are manually added by contributors. Emoji list is manually compiled from multiple sources.

Thank you! 🙏

@muan Excuse me, but I'm curious about how names and emoji codes are manually matched. I want to know the source :blush:
Do you get the supported emoji from emoji.json
Do you get the corresponding emoji code from emoji-cheat-sheet

@pageYe123  Both supported emoji & code are from or should match https://github.com/github/gemoji. But many of the manual match were with https://emojipedia.org/ since gemoji isn't always up to date.


Update README with codecov.io badge

Codecov Report

Merging #88 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #88   +/-   ##
=======================================
  Coverage   47.14%   47.14%
=======================================
  Files          10       10
  Lines        1016     1016
=======================================
  Hits          479      479
  Misses        537      537

Flag
Coverage Δ

#stable
47.14% <ø> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e26db3...f1eb193. Read the comment docs.


why use the reverberated speech signal as the training target

hi,
it a great amazing project, thanks for your effort.
When I looked at the code, I found that the training target signal was reverberated speech.  (https://github.com/Audio-WestlakeU/NBSS/blob/af66db92bb9d6f72f7100d613d3df38c40b10b09/data_loaders/ss_semi_online_dataset.py#L294C27-L294C27)
I wander why not use clean speech as the training target, as it would not only separate speakers, but also remove reverberation and even noise.

pls check sms_wsj_plus.py which is the latest dataset for jointly speech separation, denoising and dereverberation. The code you referred to is old and not used in SpatialNet.

thanks for your response!
i have another 2 questions regarding to the sms_wsj_plus dataset that the speech signal in this dataset is treated as babble noise source:
https://github.com/Audio-WestlakeU/NBSS/blob/e988a6ec845b6153910bbd106059a50b0b2c4a09/data_loaders/sms_wsj_plus.py#L95C9-L95C115
        self.noises = list(set(original_sources))  # take the speech signal in this dataset as babble noise source

as the babble noise is speech and the targets of the network is also speech, how can the model know which source are the targets, the babble or the other speech?
as the babble noise is directional source, does the babble noise need to be convoluted with the rirs?

thanks!

@flytair

as the babble noise is speech and the targets of the network is also speech, how can the model know which source are the targets, the babble or the other speech?

The babble noise is diffuse, while the target speech signals are directional, that is the key clue for the model to learn to distinguish them.

as the babble noise is directional source, does the babble noise need to be convoluted with the rirs?

The babble noise is diffuse not directional, so it doesn't need to be convolved with rirs. And we use the method implemented in https://github.com/Audio-WestlakeU/NBSS/blob/main/data_loaders/utils/diffuse_noise.py to make it diffuse.

thanks for your response!
do you think it is reasonable to use wham noise as babble noise in sms_wsj_plus dataset?


Is it working background or when app killed ?

I mean when app is not opened

No, It's only working when calling the once method.


fix: replace next vars to public

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Balanced columns

I saw that savvior splits columns evenly regarding the number of items in each column.
This means, that every column will have the same amount of items (plus/minus one).
However, when one of the items in the column is very very tall, taller that most other items, this creates an uneven column.
Are there any thoughts on how to overcome this? Is this something that is going to be supported by savvior in the future?
Thanks

The current logic is very simplistic indeed, the total number of items are evenly divided. The sole reason for this behaviour is that the dimensions of grid elements cannot be determined at the time a savvior grid is usually created. Images, iframes, and other non-blocking elements that load after savvior.init() can distort the even "look" of the columns. That said, at the moment, Savvior works best with items in a grid which do not vary a lot in height.
I am thinking about an alternative algorithm to distribute grid items based on their heights however this would impact the overall experience. For such even distributions to work first all elements that savvior need to act on need to be loaded, only after the grid can be created. This potentially causes a flash of layout change which I generally try to avoid.
Basically, I am open to suggestions here.

Can't think of anything. We are experimenting with several ideas in a project at the moment, so I'll pass this around to the rest of my team to see if we come up with something.

I encountered the same issue (otherwise I really love this lib, because it's so fast) and tried to add a comparison of heights for the division of the items.
(The code is not meeting your coding standards and I didn't add a unit test yet, but before that I wanted to ask if you would consider something like this:)
https://github.com/manuel-berger/savvior/commit/cc321440426751ace61875f3c5f2d3959416f86e#diff-51d045e87789faf8da1798427e452a3b

Thanks for this I think it looks good. Even though it still cannot solve the problem of potential initial layout repaints with images or iframes, I would certainly consider this feature as a configurable option via savvior.init(), preferably disabled by default. I am thinking of having pluggable methods of a 'default' (the current) and 'measured' (this proposal) item distribution.
You are most welcome to prepare a fully working PR complete with test coverage and documentation, let me know if I can be of any help.

I was just encountering this issue myself and would also love the ability for even spread out elements based on their height. Any idea when/if this got added?
Thanks,

Maybe you could use the code I linked above. I have to read into the JS-unit tests, config-option and documentation, etc. to make a decent pull request. Unfortunately I can't do it this week, maybe in about 2 weeks. But maybe you can use the code in the meantime.

Any news?

The sole reason for this behaviour is that the dimensions of grid elements cannot be determined at the time a savvior grid is usually created.

The way I usually do it is to calculate the image ratio server-side and then attach it with a data-ratio attribute to the img element. Then, it’s pretty easy to calculate its final size even before it is loaded. Might that be a strategy for savvior?

I shall keep this open for future reference but I cannot dedicate much time to implement new features on this right now. I guess we'd need to update the tooling around this library in order to make it easier to contribute.

This project has not received any updates for several years now. Feel free to fork this project if you still rely on it. I am closing this issue now, thank you for your continued care!


Should SQLite autoincrement be enabled?

https://github.com/vapor/sqlite-kit/blob/f0980810d474d092c86479c656cb50754c1e10c7/Sources/SQLiteKit/SQLiteDialect.swift#L37-L39
Is this dialect definition correct? SQLite supports autoincrement for integer primary keys, so I feel like this should be true and not false.
I ran into this today trying to reuse a Model in a SQLite environment, and was blocked by the 'Autoincrement not supported, skipping' error path.

Duplicate of https://github.com/vapor/sqlite-kit/issues/72.


Load census data for the 4 territories (e.g., the CSVs with income data)

The data for the Island Areas is collected through the Decennial Census of Island Areas.
Each individual territory has its own Summary File, but it's not SF1.
There's the American Samoa Summary File, the Guam Summary File, the Commonwealth of the Northern Mariana Islands Summary File, and the US Virgin Islands Summary File:

https://api.census.gov/data/2010/dec/as.html
https://api.census.gov/data/2010/dec/gu.html
https://api.census.gov/data/2010/dec/mp.html
https://api.census.gov/data/2010/dec/vi.html

It looks like some of these have a separate Year of Entry File available. I did a quick check and it looks like the data is available down to the block level.
We would want to load income and high school graduation variables to start.
This includes just adding these as data sources for the ETL and their associated tests, not adding to the score and file generation.

@lucasmbrown-usds @VincentLaUSDS @esfoobar-usds lt's touch base about this today or tomorrow on slack


OOME/Excessively deep type on upgrade from 4.9.4 to 5.x, introduced in 5.0.0-dev.20230203

Bug Report
🔎 Search Terms
tsc 5.x Out of Memory Heap
🕗 Version & Regression Information

This is a crash
This changed between versions 5.0.0-dev.20230202 and 5.0.0-dev.20230203

⏯ Playground Link
Unavailable, as I'm trying to compile an internal codebase, but I can provide access to our github repo if possible.
💻 Code
The "Type is excessively deep" is being triggered in code that calls this function:
export type LoadedCollaboration = Loaded<Collaboration, "parent">;
export async function getCollaborationMetadata(c: LoadedCollaboration): Promise<CollaborationMetadata> {
  ...
}

Where Loaded is a mapped type (which is available in our open source project):
https://github.com/stephenh/joist-ts/blob/main/packages/orm/src/loadHints.ts#L168
And the Collaboration "parent" key (from our internal project) is a union type with ~20 different types in it:
export type CollaborationParent =
  | ScheduleSubPhase
  | Approval
  ...15 others...
  | ToDo
  | SchedulePhase;

🙁 Actual behavior
In TS 4.9.4, our codebase compiled fine, in about 50 seconds.
I tried upgrading to TS 5.1.3, and the compile now OOMEs, and if given enough RAM, eventually after ~10-15 minutes, fails with a "Type instantiation is excessively deep and possibly infinite" error.
By using the 5.0.0 nightlies published to npm, I was able to bisect that version 5.0.0-dev.20230203 is what introducing the issue, and caused our compile times to jump from ~50 seconds -> 10+ minutes & fail with the type error.
🙂 Expected behavior

Hi @jakebailey , hope the tag is okay, I saw your note on #53087 that any non-4.8 -> 4.9 OOME issue should be failed as a new bug.
This one is not 5.0 -> 5.1, but 4.9.4 -> 5.0.2 (with the specific nightly identified in the ticket description).
I haven't tried running diagnostics yet, b/c I kind of assumed that would fail, but I'll try that and see what happens. Thank you!

Here's a trace (I haven't looked at it, just attaching it--generated with tar -Jcvf to make it below the github upload limit), and the compile error when giving 16gb of ram:
src/utils/collaborationMetadata.test.ts:33:39 - error TS2589: Type instantiation is excessively deep and possibly infinite.

33       const collaborationMeta = await getCollaborationMetadata(collaboration);
                                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

src/utils/collaborationMetadata.test.ts:33:64 - error TS2345: Argument of type 'Loaded<Collaboration, DeepLoadHint<Collaboration>>' is not assignable to parameter of type 'LoadedCollaboration'.
  The types of 'parent[RelationU]' are incompatible between these types.
    Type '(Task & { collaboration: LoadedOneToOneReference<CollaborationParent, Loaded<Collaboration, DeepLoadHint<Collaboration>>>; }) | ... 243 more ... | (Product & ... 1 more ... & { ...; })' is not assignable to type '(Task & {}) | (Project & {}) | (Document & {}) | (Approval & {}) | (Bill & {}) | (ToDo & {}) | (DevelopmentCommitment & {}) | ... 237 more ... | (Product & ... 1 more ... & {})'.
      Type 'Task & BidContract & { collaboration: LoadedOneToOneReference<CollaborationParent, Loaded<Collaboration, DeepLoadHint<Collaboration>>>; }' is not assignable to type '(Task & {}) | (Project & {}) | (Document & {}) | (Approval & {}) | (Bill & {}) | (ToDo & {}) | (DevelopmentCommitment & {}) | ... 237 more ... | (Product & ... 1 more ... & {})'.
        Type 'Task & BidContract & { collaboration: LoadedOneToOneReference<CollaborationParent, Loaded<Collaboration, DeepLoadHint<Collaboration>>>; }' is not assignable to type 'BidContract & Task & {}'.
          Types of property 'parent' are incompatible.
            Type 'Reference<Task, ScheduleParent, never> & ManyToOneReference<BidContract, Development, never>' is not assignable to type 'ManyToOneReference<BidContract, Development, never> & Reference<Task, ScheduleParent, never>'.
              Type 'Reference<Task, ScheduleParent, never> & ManyToOneReference<BidContract, Development, never>' is missing the following properties from type 'ManyToOneReference<BidContract, Development, never>': [RelationT], [RelationU]

33       const collaborationMeta = await getCollaborationMetadata(collaboration);
                                                                  ~~~~~~~~~~~~~

trace2.tar.gz

we have this issue as well, upgrading from 4.9 to 5+  breaks our tsc build with an eventual heap allocation error

Updated from 4.9.5 to 5.1.3 today and having the same exact issue. Build hangs until it runs OOM. Using references and paths in a Yarn PnP monorepo.

Same here... 5.0.4 used to work fine after upgrading to the current latest version 5.1.3. Very slow builds and eventually crashes. My root-cause seems to be a circular issue of some sort with tsc.
Seems like tsc can't handle interface definition files anymore, in this case from lodash?
node_modules/@types/lodash/common/common.d.ts:194:15 - error TS2589: Type instantiation is excessively deep and possibly infinite.

194     interface Object<T> extends LoDashImplicitWrapper<T> {
                  ~~~~~~

node_modules/@types/lodash/common/common.d.ts:206:15 - error TS2430: Interface 'ObjectChain<T>' incorrectly extends interface 'LoDashExplicitWrapper<T>'.
  The types returned by 'entries().pop()' are incompatible between these types.
    Type 'CollectionChain<string | T[keyof T]>' is missing the following properties from type 'ObjectChain<[string, any]>': assign, assignIn, assignInWith, assignWith, and 11 more.

206     interface ObjectChain<T> extends LoDashExplicitWrapper<T> {
                  ~~~~~~~~~~~

My workaround was adding "skipLibCheck": true to the tsconfig.json did mitigate the issue for now.

I bisected this to: https://github.com/microsoft/TypeScript/pull/52392 and managed to get a (somewhat) minimal repro case. It might still have some red herrings in it but well, it's still worth sharing 😉

300 LOC repro
type MaybeBaseType = any;
interface Flavoring<FlavorT> {
  _type?: FlavorT;
}
type Flavor<T, FlavorT> = T & Flavoring<FlavorT>;
type IdOf<T> = T extends {
  id: infer I | undefined;
}
  ? I
  : never;

type ValueFilter<V, N> =
  | V
  | V[]
  | N
  | undefined
  | {
      eq: V | N | undefined;
    }
  | {
      ne: V | N | undefined;
    }
  | {
      in: (V | N)[] | undefined;
    }
  | {
      nin: (V | N)[] | undefined;
    }
  | {
      gt: V | undefined;
    }
  | {
      gte: V | undefined;
    }
  | {
      lt: V | undefined;
    }
  | {
      lte: V | undefined;
    }
  | {
      like: V | undefined;
    }
  | {
      ilike: V | undefined;
    }
  | {
      contains: V | undefined;
    }
  | {
      overlaps: V | undefined;
    }
  | {
      containedBy: V | undefined;
    }
  | {
      gte: V | undefined;
      lte: V | undefined;
    }
  | {
      between: [V, V] | undefined;
    };

interface Collection<T extends Entity, U extends Entity>
  extends Relation<T, U> {
  load(opts?: { withDeleted: boolean }): Promise<ReadonlyArray<U>>;
  find(id: IdOf<U>): Promise<U | undefined>;
  includes(other: U): Promise<boolean>;
  add(other: U): void;
  remove(other: U): void;
  readonly isLoaded: boolean;
}

type SuffixSeperator = ":" | "_";
type DropSuffix<K> = K extends `${infer key}${SuffixSeperator}ro` ? key : K;
type NormalizeHint<T extends Entity, H> = H extends string
  ? Record<DropSuffix<H>, {}>
  : H extends ReadonlyArray<any>
  ? Record<DropSuffix<H[number]>, {}>
  : {
      [K in keyof H as DropSuffix<K>]: H[K];
    };
declare const I: unique symbol;
interface AsyncProperty<T extends Entity, V> {
  isLoaded: boolean;
  load(): Promise<V>;
  [I]?: T;
}
type LoadableValue<V> = V extends Reference<any, infer U, any>
  ? U
  : V extends Collection<any, infer U>
  ? U
  : V extends AsyncProperty<any, infer P>
  ? P extends infer U extends Entity | undefined
    ? U
    : P
  : never;
type Loadable<T extends Entity> = {
  -readonly [K in keyof T as LoadableValue<T[K]> extends never
    ? never
    : K]: LoadableValue<T[K]>;
};
interface LoadedReference<
  T extends Entity,
  U extends Entity,
  N extends never | undefined
> extends Reference<T, U, N> {
  id: IdOf<U> | N;
  get: U | N;
  getWithDeleted: U | N;
  isSet: boolean;
}
interface LoadedOneToOneReference<T extends Entity, U extends Entity>
  extends LoadedReference<T, U, undefined> {
  get: U | undefined;
  getWithDeleted: U | undefined;
  idOrFail: IdOf<U>;
  idUntagged: string | undefined;
  idUntaggedOrFail: string;
  readonly isSet: boolean;
}
interface LoadedCollection<T extends Entity, U extends Entity>
  extends Collection<T, U> {
  get: ReadonlyArray<U>;
  getWithDeleted: ReadonlyArray<U>;
  set(values: U[]): void;
  removeAll(): void;
}
interface LoadedProperty<T extends Entity, V> {
  get: V;
}

type LoadHint<T extends Entity> =
  | (keyof Loadable<T> & string)
  | ReadonlyArray<keyof Loadable<T> & string>
  | NestedLoadHint<T>;

type NestedLoadHint<T extends Entity> = {
  [K in keyof Loadable<T>]?: Loadable<T>[K] extends infer U extends Entity
    ? LoadHint<U>
    : {};
};

declare const deepLoad: unique symbol;
type DeepLoadHint<T extends Entity> = NestedLoadHint<T> & {
  [deepLoad]: true;
};

type MarkDeepLoaded<T extends Entity, P> = P extends OneToOneReference<
  MaybeBaseType,
  infer U
>
  ? LoadedOneToOneReference<T, Loaded<U, DeepLoadHint<U>>>
  : P extends Reference<MaybeBaseType, infer U, infer N>
  ? LoadedReference<T, Loaded<U, DeepLoadHint<U>>, N>
  : P extends Collection<MaybeBaseType, infer U>
  ? LoadedCollection<T, Loaded<U, DeepLoadHint<U>>>
  : P extends AsyncProperty<MaybeBaseType, infer V>
  ? [V] extends [(infer U extends Entity) | undefined]
    ? LoadedProperty<T, Loaded<U, DeepLoadHint<U>> | Exclude<V, U>>
    : V extends readonly (infer U extends Entity)[]
    ? LoadedProperty<T, Loaded<U, DeepLoadHint<U>>[]>
    : LoadedProperty<T, V>
  : unknown;

type MarkLoaded<T extends Entity, P, UH = {}> = P extends OneToOneReference<
  MaybeBaseType,
  infer U
>
  ? LoadedOneToOneReference<T, Loaded<U, UH>>
  : P extends Reference<MaybeBaseType, infer U, infer N>
  ? LoadedReference<T, Loaded<U, UH>, N>
  : P extends Collection<MaybeBaseType, infer U>
  ? LoadedCollection<T, Loaded<U, UH>>
  : P extends AsyncProperty<MaybeBaseType, infer V>
  ? [V] extends [(infer U extends Entity) | undefined]
    ? LoadedProperty<T, Loaded<U, UH> | Exclude<V, U>>
    : V extends readonly (infer U extends Entity)[]
    ? LoadedProperty<T, Loaded<U, UH>[]>
    : LoadedProperty<T, V>
  : unknown;

type Loaded<T extends Entity, H> = T & {
  [K in keyof T & keyof NormalizeHint<T, H>]: H extends DeepLoadHint<T>
    ? MarkDeepLoaded<T, T[K]>
    : MarkLoaded<T, T[K], NormalizeHint<T, H>[K]>;
};

interface Entity {}

declare const RelationT: unique symbol;
declare const RelationU: unique symbol;
interface Relation<T extends Entity, U extends Entity> {
  [RelationT]: T;
  [RelationU]: U;
  isLoaded: boolean;
}

declare const ReferenceN: unique symbol;
interface Reference<
  T extends Entity,
  U extends Entity,
  N extends never | undefined
> extends Relation<T, U> {
  readonly isLoaded: boolean;
  load(opts?: { withDeleted?: boolean; forceReload?: true }): Promise<U | N>;
  set(other: U | N): void;
  [ReferenceN]: N;
}
declare const OneToOne: unique symbol;
interface OneToOneReference<T extends Entity, U extends Entity>
  extends Reference<T, U, undefined> {
  [OneToOne]: T;
}

interface PolymorphicReference<
  T extends Entity,
  U extends Entity,
  N extends never | undefined
> extends Reference<T, U, N> {
  id: IdOf<U> | undefined;
  idOrFail: IdOf<U>;
  idUntagged: string | undefined;
  idUntaggedOrFail: string;
  readonly isSet: boolean;
}

type BidContractId = Flavor<string, BidContract>;

interface BidContractFilter {
  id?: ValueFilter<BidContractId, never>;
}

declare abstract class BidContractCodegen {
  static readonly tagName = "bc";

  readonly __orm: {
    filterType: BidContractFilter;
  };

  readonly collaboration: OneToOneReference<BidContract, Collaboration>;
}

class BidContract extends BidContractCodegen {}

type BidItemId = Flavor<string, BidItem>;

interface BidItemFilter {
  id?: ValueFilter<BidItemId, never>;
}

declare abstract class BidItemCodegen {
  static readonly tagName = "bi";

  readonly __orm: {
    filterType: BidItemFilter;
  };

  readonly collaboration: OneToOneReference<BidItem, Collaboration>;
}

declare class BidItem extends BidItemCodegen {}

type BillId = Flavor<string, Bill>;

interface BillFilter {
  id?: ValueFilter<BillId, never>;
}

declare abstract class BillCodegen {
  static readonly tagName = "b";

  readonly __orm: {
    filterType: BillFilter;
  };

  readonly collaboration: OneToOneReference<Bill, Collaboration>;
}

declare class Bill extends BillCodegen {}

// original union contained 19 types, might be worth introducing more here to stress test this
export type CollaborationParent = BidContract | BidItem | Bill;

declare abstract class CollaborationCodegen {
  static readonly tagName = "collab";
  readonly parent: PolymorphicReference<
    Collaboration,
    CollaborationParent,
    never
  >;
}

class Collaboration extends CollaborationCodegen {}

type LoadedCollaboration = Loaded<Collaboration, "parent">;

type CollaborationMetadata = {
  id: string;
  projectId: string | undefined;
  name: string;
  blueprintPath: string;
  type: string;
};

declare function getCollaborationMetadata(
  c: LoadedCollaboration
): Promise<CollaborationMetadata>;

declare const collaboration: Loaded<Collaboration, DeepLoadHint<Collaboration>>;

getCollaborationMetadata(collaboration);

export {};

I can't exactly end up with OOM using this repro but It just can't finish up running. So perhaps this is an infinite loop and this small extracted repro just can't push the whole thing over the edge when it comes to memory.

I tried the above code; it takes a good 15 seconds to run and uses 800MB of memory, but it still completes and doesn't OOM. It's an interesting case but I'm not 100% sure if it's representative of the original issue. The timing is right based on the issue title, though, so it probably is good enough to start looking into this.

FWIW, I recently published a tool which should make it a lot easier to figure out what changed things: https://www.npmjs.com/package/every-ts#bisecting
@stephenh (and others in this thread), it'd be super helpful if you could use this tool to identify what change actually caused the problem.

In fact, main now declines to compile that code:
index.ts:312:1 - error TS2859: Excessive complexity comparing types 'Loaded<Collaboration, DeepLoadHint<Collaboration>>' and 'LoadedCollaboration'.

312 getCollaborationMetadata(collaboration);
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

index.ts:312:26 - error TS2345: Argument of type 'Loaded<Collaboration, DeepLoadHint<Collaboration>>' is not assignable to parameter of type 'LoadedCollaboration'.

312 getCollaborationMetadata(collaboration);
                             ~~~~~~~~~~~~~

Found 2 errors in the same file, starting at: index.ts:312

I tried the above code; it takes a good 15 seconds to run and uses 800MB of memory, but it still completes and doesn't OOM.

Interesting. When reducing this test case from their private repo I could make it to finish - I didn't check the mem usage levels. It looked like an infinite loop that couldn't finish. I think I was still OOMing this within the repo but when executed standalone it was just hanging for a loooong time.

@stephenh (and others in this thread), it'd be super helpful if you could use this tool to identify what change actually caused the problem.

I already bisected this to https://github.com/microsoft/TypeScript/pull/52392

Yeah, sorry, I noticed but forgot to change my text to say "anyone else with this problem"!


call_method*() crashes when the method does not exist

extern crate pyo3;

use pyo3::{Python, ObjectProtocol};

fn main() {
    let gil = Python::acquire_gil();
    let py = gil.python();
    let a = py.eval("42", None, None).unwrap();
    a.call_method0("__str__").unwrap();  // ok
    a.call_method0("nonexistent_method").unwrap();  // crashes
}

Ideally should produce AttributeError or something.

This happens because in https://github.com/PyO3/pyo3/blob/30baa990f8d1ec68bf60354eef0151c9c07ce269/src/objectprotocol.rs#L329-L340 and several similar places, ptr could be NULL.

we should return PyErr if ptr is null,
https://github.com/PyO3/pyo3/blob/master/src/err.rs#L239

fixed in master


feat: Random number generation

In advanced concepts chapter, with security concerns.

Hello @julio4 i'll like to be assigned to this task

I'm Sorry i can't handle this for now, thank you.

@julio4 can you assign this to me.


Fix obsolete client calls for contentful.js 3.x

roots-contentful updated to contentful.js 3.x in April 2016, but the code still invokes old method names that no longer exist in contentful.js. The resulting error message is:

TypeError: client.entries is not a function

This change updates the client calls from client.entries() to client.getEntries(), and from client.contentType to client..getContentType.

lgtm


[Question] Chunks from transcription always empty text

This example works fine:

But ATM I am sending Float32 to the worker here (i also confirm the audio is valid by playing it back)
https://github.com/quantuminformation/coherency/blob/main/components/audio-recorder.js#L104
But after transcribing here:
https://github.com/quantuminformation/coherency/blob/main/worker.js#L140
my chunks only contain ""

any ideas where my setup is going wrong?

Are you setting the sampling rate to 16000Hz? That might be something which is missing.

I think you can fix it by setting sampleRate here:
https://github.com/quantuminformation/coherency/blob/fb259ce35547aef8e4ad894fdcb2d6b6d637ed6d/components/audio-recorder.js#L20
e.g.,
const audioContext = new (window.AudioContext || window.webkitAudioContext)({
    sampleRate: Constants.SAMPLING_RATE,
});

Are you setting the sampling rate to 16000Hz? That might be something which is missing. If I remember correctly, it defaults to 44.1kHz or 48kHz, so you need to override this.
e.g., in whisper-web:
https://github.com/xenova/whisper-web/blob/653f9790bd4cb9e23f6e094c4ba9454b190fe3fe/src/components/AudioManager.tsx#L182-L184

yeah, using 16000 : https://github.com/quantuminformation/coherency/blob/main/constants.js#L31
btw what do you think about the skipping react and only using vanilla

yeah, using 16000 : https://github.com/quantuminformation/coherency/blob/main/constants.js#L31

But are you using it? The only place it is used is in routes/process-map.js

yeah thx, fixed that, same issue, must be smth else im missing

https://github.com/quantuminformation/coherency/commit/f046c0403d65469e168a2d647a32c2aa039d5c65

I also added the missing blob fix, but still get no chunks back except ""
I wonder if there is an issue with cdn version, maybe I should try npm?

I wonder if there is an issue with cdn version, maybe I should try npm?

the CDN and npm versions should give the same result. If you're still stuck, perhaps you can work backwards from whisper web, since the audio processing is done correctly there.

ill probably do that thanks, i think the project is on hold now due to my delays


Issue when using num_complex with assert_relative_eq

Hi,
My formal gist is working really well. Unfortunately, we have made some change in our dependencies and now we don't use the crate num anymore, but we rely on num_complex. Here is the gist to reproduce my error.
I have used the save num-complex version as your library but I face this error when I do a cargo build:
error[E0277]: the trait bound `num_complex::Complex<{float}>: approx::RelativeEq` is not satisfied
  --> src/main.rs:28:5
   |
28 |     assert_relative_eq!(Complex::new(0.2, 0.1), Complex::new(0.2, 0.1));
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `approx::RelativeEq` is not implemented for `num_complex::Complex<{float}>`
   |
   = note: required by `approx::Relative`
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0277]: the trait bound `num_complex::Complex<{float}>: approx::RelativeEq` is not satisfied
  --> src/main.rs:28:5
   |
28 |     assert_relative_eq!(Complex::new(0.2, 0.1), Complex::new(0.2, 0.1));
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `approx::RelativeEq` is not implemented for `num_complex::Complex<{float}>`
   |
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error[E0277]: the trait bound `num_complex::Complex<f64>: approx::RelativeEq` is not satisfied
  --> src/main.rs:29:5
   |
29 |     assert_relative_eq!(sh.as_slice().unwrap(), gt.as_slice().unwrap());
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ the trait `approx::RelativeEq` is not implemented for `num_complex::Complex<f64>`
   |
   = note: required because of the requirements on the impl of `approx::RelativeEq` for `[num_complex::Complex<f64>]`
   = note: required because of the requirements on the impl of `approx::RelativeEq` for `&[num_complex::Complex<f64>]`
   = note: required by `approx::Relative`
   = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

error: aborting due to 3 previous errors

I have checked and you use the num-complex type so I don't get why it's not working. Did I miss something?

I have made the update to 0.3.0 and everything is fine. I close it.

Awesome, glad I made that release now! 😅


98 dataset elegibility criteria

Proposed Changes

Able the next button in Elegibility criteria form if there are data or not.

Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Tests
[x] Build locally
[ ] Code format / lint
[ ] Docs have been reviewed and added

Pull request type
Please check the type of change your PR introduces:

[ ] Bugfix
[x] Feature
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Other (please describe):

What is the current behavior?

In the form of Data set, in the Elegibility criteria section, the next button is disabled if there are not registries.

Fixes #98
What is the new behavior?

Now the next button is enabled if there are not registries.

In the template of DataSetCreation component, the "next" button had a conditional, if elegibilityCriteriaList have any data, the button will be enabled, but this conditional was deleted and now the button is enabled ever.

Does this introduce a breaking change?

[ ] Yes
[x] No

Codecov Report

Merging #100 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master    #100   +/-   ##
======================================
  Coverage    6.99%   6.99%
======================================
  Files          23      23
  Lines         386     386
  Branches       27      27
======================================
  Hits           27      27
  Misses        359     359

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c868e82...c7676c3. Read the comment docs.


Pick the onnx.utils.extract_model()

It will be easy to maintain here. And we don't need to depend on a specific version of ONNX.

Codecov Report

Merging #5 (c7f9b24) into master (230f2ec) will increase coverage by 0.31%.
The diff coverage is 93.82%.

@@            Coverage Diff             @@
##           master       #5      +/-   ##
==========================================
+ Coverage   92.21%   92.53%   +0.31%
==========================================
  Files           7        7
  Lines         257      335      +78
==========================================
+ Hits          237      310      +73
- Misses         20       25       +5

Impacted Files
Coverage Δ

onnxcli/extract.py
94.56% <93.82%> (-5.44%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 230f2ec...c7f9b24. Read the comment docs.


fix: NRE in WorkingCopy (#199)

It seems that the Silicon compilation failure is a bug in the .net SDK or github actions?


Another logic for handling interval of fade_brightness() within init.py

Previously, interval was defined as the time delay between the end of one brightness change and the start of the next. This meant that the actual time between brightness changes was always longer than the specified interval, as it included the time taken by the brightness change itself. This resulted in the total fading time being longer than interval*steps, with the deviation increasing with more steps. This resulted in unpredictable time consumption.
This PR adds a new parameter strict_interval to the fade_brightness(). interval  can now be interpreted as the intended time between the start of each brightness change. However, if a brightness change takes longer than the specified interval, the actual interval will be longer, as the next change can't start until the current one finishes.
test result:

start=1		finish=21
-----------------
interval=0.01s	expected=0.2s
before=1.94s	deviation=1.74s
after=1.46s	deviation=1.26s
time_saved_percent='239.95%'
-----------------
interval=0.05s	expected=1.0s
before=2.58s	deviation=1.58s
after=1.63s	deviation=0.63s
time_saved_percent='103.58%'
-----------------
interval=0.1s	expected=2.0s
before=3.70s	deviation=1.70s
after=2.19s	deviation=0.19s
time_saved_percent='75.81%'
-----------------
interval=0.5s	expected=10.0s
before=11.62s	deviation=1.62s
after=10.19s	deviation=0.19s
time_saved_percent='14.25%'
-----------------
interval=1s	expected=20s
before=22.17s	deviation=2.17s
after=20.21s	deviation=0.21s
time_saved_percent='9.81%'
-----------------

test code:
import screen_brightness_control as sbc
import time

start = 1       # included
finish = 21     # included
intervals = [0.01, 0.05, 0.1, 0.5, 1]
print(f"{start=}\t\t{finish=}")
print("-----------------")

for interval in intervals:
    expected = interval*abs(finish-start)
    print(f"{interval=}s\t{expected=}s")
    kwargs = {"finish": finish, "start": start, "increment": 1, "interval": interval, "logarithmic": False, "display": 0}

    t1 = time.time()
    kwargs["strict_interval"] = False
    sbc.fade_brightness(**kwargs)
    t2 = time.time()
    before = t2-t1
    deviation = before-expected
    print(f"{before=:.2f}s\t{deviation=:.2f}s")

    kwargs["strict_interval"] = True
    sbc.fade_brightness(**kwargs)
    t3 = time.time()
    after = t3-t2
    deviation = after-expected
    print(f" {after=:.2f}s\t{deviation=:.2f}s")

    time_saved = before-after
    time_saved_percent = f"{time_saved/expected*100:.2f}%" if expected != 0 else "can not be calculated"
    print(f"{time_saved_percent=}")
    print("-----------------")

This change makes the time consumption of fade_brightness() more predictable. Note that this PR is not intended to make the time consumption equal to the expectation, but to bring it closer.
Furthermore, I recommend removing the strict_interval and making this new interval handling logic the default and only method.

Thanks for this. Makes alot of sense.

Furthermore, I recommend removing the strict_interval and making this new interval handling logic the default and only method.

I agree. Can't see much reason for keeping the old behaviour around


Support for photo-z bias parameters

We need to add a modelling of photoz biases in order to replicate a simple DES Y1 analysis

Done


vimPlugins.vim-visual-star-search: init at 2021-07-14

Description of changes
Yes, there's already a vim package: vim-visualstar, though it's code pretty much opinionated with yanking a selected text to a main register. Current plugin is much simpler and more recent.

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/1785

please don't immediately post this on the discourse thread after opening the pull request, at least wait for a day or two


Duplicate symbol declaration

VSCode Version: 1.32.1 (user setup)
OS Version: Windows_NT x64 10.0.17763

Steps to Reproduce:
Say there's a code like the following :
if(condition) {
    define('SOMETHING','value);
}
else {
    define('SOMETHING','value');
}

I get the error Duplicate symbol declaration  but they're not duplicates, as there's an if condition in between.

Does this issue occur when all extensions are disabled?: No

caused by extension https://github.com/bmewburn/vscode-intelephense/issues/320


Feedback for Elements in the Screen Designer

We have recently added a grid component as an element that should be documented. It has a bit more advanced set of capabilities like width adjustment etc..

Document Details
⚠ Do not edit this section. It is required for docs.superOffice.com ➟ Docs Team processing.

Content Source: help-en-screen-designer-elements

Thanks for the reminder @ErikEide .


[twelvehour]=true not working

i set this to true but no AM or PM option available

https://github.com/kuhnroyal/mat-datetimepicker/blob/23235be45057f0104115afa5a8a2dc7fc0e4f38f/projects/core/src/datetimepicker/clock.ts#L107

Yea, I saw this as well. This is still left over from the original date-time-picker but even there it was not a working input. I will see what I can do.

Does this mean it is not possible to show the clock in 12 hour mode in any way?

Can we show the clock in 12 hour mode?

I created the pull request with the 12 hour feacture, but I am waiting for approval.
https://github.com/kuhnroyal/mat-datetimepicker/pull/97


Monorepo setup issue

Description
I have a yarn workspaces based mono-repo. The code is located at https://github.com/purplemana/alchemist. It's a TypeScript project. There are multiple modules under packages. Every module has its own tsconfig.json. Specs are co-located with the source files with *.test.ts extension. The most complex module is api-server, where I have defined paths in tsconfig.json to resolve synthetic imports. To make the compiled JS code run, I use tsconfig-paths module with mocha, e.g.
"test": "yarn build && cross-env NODE_ENV=test mocha -r tsconfig-paths/register ./build/**/*.test.js"

For equivalent configuration, I have tried to configure the extension with these settings:
mocha path: mocha/lib/mocha.js
test files location: packages/*/build/**/*.test.js
files to ignore: **/.git/**/*,**/node_modules/**/*
environmets: {"NODE_ENV":"test"}
requires: ["tsconfig-paths/register"]
options:  {}

When I run the tests from the extension, I get this error -
Error: Cannot find module 'fixtures/db'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (c:\code\alchemist\packages\api-server\build\src\core-api\auth\createUser.test.js:23:14)

My guess is that this is because mocha is not being run relative to each module's tsconfig.json location, e.g. packages/api-server. This is causing the tsconfig-paths module to not be able to resolve the synthetic paths.
How can I make this extension work for my setup?
Thanks
Versions

VScode: v1.28.0
Mocha SideBar: v0.20.18
Node Version: v10.6.0

OS version

[ ] Linux
[ ] Mac
[x] Windows
[x] Windows Subsystem for Linux running Ubuntu

I'll try to check it for you guys

@maty21 btw, for some reason chai/register-assert is not found during registration, so question, is this monorepo too, or this plugin is designed in such a way that it uses own node_modules for dependencies like Babel during starting mocha? Notice for instance that I use Babel 7 and this plugin example suggests using older Babel which doesnt use namespace @babel yet. If thats a separate issue I could create one, unless it has sth to do with monorepo as well. Anyway, thx for awesome plugin, if those issues are solved, I will get even bigger motivation to write tests ;)

I don't think that that is your problem imo it's probably realted to a
incorrect setting . What is the mocha cli command you running and working
for you
On Fri, Nov 2, 2018, 10:32 AM Konrad Lisiczyński<EMAIL_ADDRESS>wrote:

@maty21 https://github.com/maty21 btw, for some reason
chai/register-assert is not found during registration, so question, is
this monorepo too, or this plugin is designed in such a way that it uses
own node_modules for dependencies like Babel during starting mocha?
Notice for instance that I use Babel 7 and this plugin example suggests
using older Babel which doesnt use namespace @babel
https://github.com/babel yet. If thats a separate issue I could create
one, unless it has sth to do with monorepo as well. Anyway, thx for awesome
plugin, if those issues are solved, I will get even bigger motivation to
write tests ;)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/maty21/mocha-sidebar/issues/152#issuecomment-435307012,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE_nuQ8Y0LV0dJGHgXq5HX9S31kDpateks5urAMGgaJpZM4XXeRA
.

@maty21 with those settings:
  "mocha.files.glob": "src/**/*.spec.js",
  "mocha.requires": [
    "@babel/register",
    "chai/register-assert"
  ],
  "mocha.env": {
    "BABEL_ENV": "test"
  },
  "mocha.coverage": {
    "enable": false,
    "decoration": false,
    "runWithInterval": false,
    "autoUpdateInterval": 20000,
    "activeOnStart": false,
    "runAfterTest": false,
    "runCoverageAfterFileSave": false,
    "reporters": []
  },
  "mocha.subdirectory": "/home/klis87/projects/redux-saga-requests/packages/redux-saga-requests"

I can see in output:
test runs with those args:

  files: [],
  options: {},
  grep:null,
  requires: ["@babel/register","chai/register-assert"],
  rootpath: "/home/klis87/projects/redux-saga-requests/packages/redux-saga-requests"

So for some reason even my tests are not found, when I try to update mocha path, it doesnt work at all, if I try to mess with mocha path too then it doesnt work at all, for example this command mocha-maty.autoPlay not found

and for coverage commend:
coverage running the following command /home/klis87/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.20/node_modules/nyc/bin/nyc.js --all  --reporter=html --reporter=text --reporter=lcov  /home/klis87/.vscode/extensions/maty.vscode-mocha-sidebar-0.20.20/node_modules/mocha/bin/mocha --require @babel/register --require chai/register-assert  src/**/*.spec.js --exit

Maybe if thats help too, I use following babel config:
module.exports = {
  presets: [
    [
      '@babel/preset-env',
      {
        loose: true,
        modules: process.env.BABEL_ENV === 'es' ? false : 'commonjs',
      },
    ],
  ],
  plugins: [
    process.env.BABEL_ENV === 'coverage' && 'istanbul',
    ['@babel/plugin-proposal-object-rest-spread', { loose: true }],
    [
      '@babel/plugin-transform-runtime',
      {
        corejs: false,
        helpers: true,
        regenerator: true,
      },
    ],
  ].filter(Boolean),
};

generally I think this plugin should NEVER use its own version of mocha etc, it should work only if user has those dependencies installed for a project. This is like eslint plugin works for instance. If someone needs plugin for mocha, one will have this installed already, so there is no use for plugins's mocha version as fallback imho, as when someone doesnt have this installed, it means a given project doesnt use mocha and this plugin shouldn't even be active.

@maty21 I guess I misunderstood your question, apologies. for testing my command in package.json in a package dir I use: "test": "cross-env BABEL_ENV=test mocha -r @babel/register -r chai/register-assert 'src/**/*.spec.js'"

Tks man if you have a sample of project that not working it will be nice so
I will able to debug it for you
On Fri, Nov 2, 2018, 11:54 AM Konrad Lisiczyński<EMAIL_ADDRESS>wrote:

@maty21 https://github.com/maty21 I guess I misunderstood your
question, apologies. for testing my command in package.json in a package
dir I use: "test": "cross-env BABEL_ENV=test mocha -r @babel/register -r
chai/register-assert 'src/**/*.spec.js'"
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/maty21/mocha-sidebar/issues/152#issuecomment-435327410,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AE_nuazLq8b-tLNejAWe8kCFoLFEMzUhks5urBZhgaJpZM4XXeRA
.

Ok, I am not sure I will be able to do this today, if not over the weekend, I will try to your your plugin in simpler scenarios, adding more and more complex and then I will give you the link for the repo and point what is problematic. Thanks!

or you can check it wright here - https://github.com/klis87/redux-saga-requests, it is open source

@maty21 here you can see example - https://github.com/klis87/mocha-sidebar-example this is extremely simple one for now, but even this seems not working. For some reason it sometimes works sometimes it doesn't on project reload (just nothing is decorated or shown in sidebar). Also command run all tests cannot find any test to run. I you prefer, I can create a separate issue for this. Once this is fixed, I will try to add more stuff until we get monorepo working.
I use newest VsCode btw.


zlib version issue

Hello
I was trying to compile your test app and I am getting the following error. could you help?
Severity	Code	Description	Project	File	Line	Suppression State
Warning	NU1701	Package 'zlib.net 1.0.4' was restored using '.NETFramework,Version=v4.6.1, .NETFramework,Version=v4.6.2, .NETFramework,Version=v4.7, .NETFramework,Version=v4.7.1, .NETFramework,Version=v4.7.2, .NETFramework,Version=v4.8' instead of the project target framework '.NETStandard,Version=v2.0'. This package may not be fully compatible with your project.	ESPTool	E:\data\Csharp\ESPTool-main\ESPTool\ESPTool.csproj	1

I missed this issue completely,
Ill look into this.


Openstack LB improvements

In particular, makes EnsureTCPLoadBalancerDeleted idempotent, and fixes an existing bug that leaked monitors on LB delete.

OK to test
On Jun 2, 2015 12:34 AM, "Kubernetes Bot"<EMAIL_ADDRESS>wrote:

Can one of the admins verify that this patch is reasonable to test? (reply
"ok to test", or if you trust the user, reply "add to whitelist")
If this message is too spammy, please complain to ixdy.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/9105#issuecomment-107840186
.

Add to whitelist
On Jun 2, 2015 5:25 AM, "Eric Tune"<EMAIL_ADDRESS>wrote:

OK to test
On Jun 2, 2015 12:34 AM, "Kubernetes Bot"<EMAIL_ADDRESS>wrote:

Can one of the admins verify that this patch is reasonable to test?
(reply "ok to test", or if you trust the user, reply "add to whitelist")
If this message is too spammy, please complain to ixdy.
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleCloudPlatform/kubernetes/pull/9105#issuecomment-107840186
.

cc @justinsb @thockin

Thanks for the improvements, @anguslees! Can you also squash the gofmt commit when you get a chance?
This fixes #8352

I don't suppose there is a comparable change coming to make the Create logic idempotent :-) ?
Not related to this PR, but is there a reason the externalIP must be specified for OpenStack LBs (vs just creating one on-demand)?  I think that API functionality is at least deprecated, and we'd like to kill it altogether.

LGTM

@anguslees Any chance you could make the CLA bot happy?

@anguslees Have you signed the CLA?

@anguslees We need an SLA from your company before merging.

@quinton-hoole,@bgrant0607: I'm doing this work on Rackspace company time, and Rackspace has signed the Google Corporate CLA.  None of the helpful googlebot links let me explore the situation further, but I suspect googlebot only knows about my @rackspace.com address, which I don't use on github, etc.  If there was a way to get googlebot to understand this, that would be great.  Currently I repeat some version of this explanation on every Kubernetes commit.

@justinsb: huh, the externalIP is meant to be optional...
Oh, from eyeballing the code, I'm guessing the externalIP ends up as <IP_ADDRESS> or something terrible.  I'll add an explicit check for externalIP.IsUnspecified() ...

@justinsb: I think I've fixed the externalIP issue.
Re idempotent Create - I'm waiting for https://github.com/GoogleCloudPlatform/kubernetes/pull/8366 to reach a conclusion first ;)  There's actually not a lot I can change on an OpenStack LBaaS VIP without deleting+recreating anyway (only really cosmetic name/description and pool members).

Hi Justin
I've manually verified your CLA for you.  In future, to avoid the hassle you should:

Add your rackspace.com email address to your github account
Make sure that Rackspace have set up an appropriate Google Group and made you a member of it.  Rackspace should have received instructions for this along with the CLA they signed.

Thanks
Q

Hi @anguslees,
I am working on OpenStack Magnum and I am trying to get createExternalLoadBalancer to work but have not been successful on a Fedora Atomic image.  I have configured kube-apiserver and kube-controller-manager with the option --cloud_config and --cloud_provider, but they either crash or would not work properly with Neutron. Is there some instruction on how to set up a known working environment?
Thanks,
Ton Ngo


Minor upgrades to apply_oe.py

Deterministic LUT naming
Surface model selection
PRISM sensor support

Excellent additions, great to have PRISM and surface models included.  Thanks for the catch on the config channelized_radiometric_uncertainty_file assignment as well!


How to nest leaflet directive inside another directive?

I am trying to nest the leaflet directive inside another directive.
I am getting the following error: angular.js:13708 TypeError: Cannot read property 'baselayers' of undefined
It is from the line:
for (var layerName in layers.baselayers) {

This is in the link function of the leaflet directive in the ui-leaflet source (ui-leaflet.js:4109).
I do not get this error if I put my  at the top level outside of a directive.

sorry this is a duplicate of:
https://github.com/angular-ui/ui-leaflet/issues/135


🖐2024-01-19T12:00/13:00+09:00: Pacific and Strategy meeting

== Let's put the link to the invite on our Discord ==

Done!

Thank you! I am copying back to the head of this issue.

https://k96mz.github.io/20240111guessNumber/
I created the guessing the number game.
I used the HTML and CSS in the course which I take. I coded JavaScript by myself.
The goal of this game is guessing the number between 1 - 20.
数当てゲームを作成。HTML, CSSは講座のものを使用。JSのみ自分で作成。
〇rule
・Enter a number between 1-20 in the input area, and click "check" button.
・If you guess the number which is concealed by "?" correctly, you win. Every time you miss, your score will decrease.
・Click "again" button to start the game again.
〇ゲームのルール。
・1-20の間の数字を入れて「check!」をクリック
・「？」に1-20の間の数字がランダムに格納されており、それに当たれば勝ち。外れる度にScoreの数値が減少。・「Again!」ボタンを押せば再度ゲーム開始。
Do we have the meeting on discord? I couldn't get the meeting link when I clicked the upper link of discord.

So cool, @k96mz ! The meeting link is here!
https://ucla.zoom.us/j/93540731519?pwd=OG91Q2FPQktpUWF1YldVNVRMT294UT09

Created the issue for the next pacific and strategy meeting at #371. I am closing this issue now.


added support for source maps

If the webpack configuration is set to generate source maps, and a source map for the file comes into the loader, emit the map in addition to the file.
This is particularly useful for css preprocessing to a file:
loaders: [ 'file?name=[name].css', 'cssnext' ]

hmm, need to add the comment.

I will redo this.


Enable passthrough

Welcome!

[X] Yes, I've searched similar issues on GitHub and didn't find any.
[X] Yes, I've searched similar issues on the Traefik community forum and didn't find any.

What did you expect to see?
Hi, It would be really useful to be able to enable passthrough as indicated here
https://doc.traefik.io/traefik/routing/routers/#passthrough
but from the helm seem to be not possible.
Thanks

Hello @mamu-reply ,
Thanks for reaching out!
TCP routers is a dynamic configuration that has been implemented through IngressRouteTCP CRD.
To keep the repository focused on the Chart, we ask that all questions be asked in the community forum. It is pretty active, so you might find that your question has already been answered there.  If not, you can ask and get help from other community members pretty quickly.


For info, created a travis script for mac

For info, created a travis script for mac, at https://github.com/hughperkins/distro/blob/macbuild/.travis.yml
I'm not quite sure how to add it in parallel to the linux build, but just linking to it, in case useful.
It has a bit of code to upload artifact to s3 at the end, which sooner or later I will start using for my mac build scripts that use torch.  Otherwise have to build whole torch each time, which is quite slow (as far as I know travis dependency caching is broken on Mac OS X?)

(so, my torch install on mac travis is super fast now :-)  https://travis-ci.org/hughperkins/cltorch/builds/112530954#L211  1.78s for torch install :-) )


is jsoup dead or abandoned?

It's been over three months that I found a pretty serious bug #1837, and even spent a lot of time fixing it and filing a pull request, but nobody has bothered even to respond. Has the owner of jsoup abandoned the project?

Hi, no -- I've just been working on other things. Thanks for checking!


How to inject and HvH????

Yo, I wanna start HvH in Cs:go and I dont really wanna pay 20 euros a month for a paid cheat because I dont know if I will like it or not, also could someone link me a HvH config? Thanks.

you cant hvh with osiris, bad anti aim, no resolver, and legit aimbot

osiris isn't a hvh cheat, it's meant strictly for legit or closet cheating. i'd suggest you try asking on reddit r/CsgoHacks

Yo, I wanna start HvH in Cs:go and I dont really wanna pay 20 euros a month for a paid cheat because I dont know if I will like it or not, also could someone link me a HvH config?

Yo, I wanna start HvH in Cs:go and I dont really wanna pay 20 euros a month for a paid cheat because I dont know if I will like it or not, also could someone link me a HvH config?  I don't know if spinbotter has bad aa or osiris have resolver

Use rage/hvh multipaste


Automate the startup of consensus, and the sending and maintaining of bids and stakes

Fixes #27

Codecov Report

Merging #67 into master will increase coverage by 0.36%.
The diff coverage is 50%.

@@            Coverage Diff             @@
##           master      #67      +/-   ##
==========================================
+ Coverage   54.94%   55.31%   +0.36%
==========================================
  Files         118      118
  Lines        6197     6335     +138
==========================================
+ Hits         3405     3504      +99
- Misses       2197     2231      +34
- Partials      595      600       +5

Impacted Files
Coverage Δ

pkg/wallet/wallet.go
46.88% <ø> (ø)
:arrow_up:

pkg/core/chain/chain.go
33.33% <ø> (+1.98%)
:arrow_up:

pkg/p2p/wire/topics/topics.go
68.96% <ø> (ø)
:arrow_up:

pkg/config/loader.go
63.79% <100%> (ø)
:arrow_up:

pkg/p2p/wire/rpcbus.go
68.96% <33.33%> (-1.95%)
:arrow_down:

pkg/core/consensus/agreement/collector.go
89.47% <0%> (-10.53%)
:arrow_down:

pkg/core/consensus/selection/selector.go
70.93% <0%> (+2.32%)
:arrow_up:

pkg/core/consensus/agreement/broker.go
78.88% <0%> (+6.03%)
:arrow_up:

pkg/core/consensus/reduction/reducer.go
76.71% <0%> (+11.64%)
:arrow_up:

... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 168a59b...6687f19. Read the comment docs.


Images not getting verified using policies defined in ImagePolicy

Problem
Neither ImagePolicy nor ClusterImagePolicy seems to be honored while installing kubernetes pod.
What commit ID of Portieris did you experience the problem with?
8a5adaf11b748b56d3f435fb9f79d2c3aa82a109
What went wrong?
Installed Portieris 0.6.0 using helm v3.1.2. Created custom image policy yaml and applied using kubectl apply -f allow-custom.yaml
Tried installing a kubernetes pod with image matching the repository name given in allow-custom.yaml. Image is pulled without any issues, but expected signature verification failure
What should have happened differently?
Signature verification should have happened with the notary server given in allow-custom.yaml policy
How can it be reproduced?

Install helm v3.1.2
Install portieris 0.6.0
Create custom image policy yaml file given in below section
kubectl apply -f allow-custom.yaml
Create new kubernetes pod with image matching the repo name given in allow-custom.yaml

Any other relevant information
allow-custom.yaml
apiVersion: securityenforcement.admission.cloud.ibm.com/v1beta1
kind: ImagePolicy
metadata:
  name: allow-custom
spec:
   repositories:
    - name: "*"
      policy:
        trust:
          enabled: true
          trustServer: "https://notary.abc.com:4443"

kube-apiserver.yaml
   - --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook

Hi, Portieris did not support helm 3 until current head, we will make a release soon where we note that helm 3 is now supported. So you are certainly in unsupported space there.
Can you check out the git repo at 0.6.0 and install with Helm 2?  (or if you dont worry about having your own certificates or changing namespace, then you can just install the charts from the release with helm2)
It could well be that the install went very badly in isn't enforcing anything.

@sjhx Thanks for your response.
Earlier I tried installing portieris with helm 2.x but faced issues in getting portieris up. https://github.com/IBM/portieris/issues/132
Recently I observed that portieris README started claiming that we can use helm 3.0 or above. Do you think I can try portieris 0.7.0 to solve this? (I am worried to go back and try with helm 2 again!)

You can try install with helm 3 from the current head, yes.

@sjhx Installed Portieris 0.7.0
Now, I am facing below issue (minor obfuscation). Is this because of https://github.com/IBM/portieris/issues/123 ?
Appreciate if I could get a suggestion to get past this error.
I0624 03:55:14.708632       1 controller.go:65] Processing admission request for CREATE on
I0624 03:55:14.708916       1 controller.go:111] Getting policy for container image: quay.io/container/myrepo:1.1.1-1   namespace: misc
I0624 03:55:14.718224       1 controller.go:191] policy.Simple {[]  }
I0624 03:55:14.718351       1 oauth.go:42] CA not provided at /etc/certs/ca.pem, will use default system pool
E0624 03:55:14.973856       1 oauth.go:111] Received non-success status code 500
E0624 03:55:14.973942       1 verifier.go:80] Request to OAuth failed with status code: 500 and body:

    <title>Internal Server Error</title>

Internal Server Error

Good you have it going.
Caused https://github.com/IBM/portieris/issues/51 and probably https://github.com/IBM/portieris/issues/123 I expect.
I hope we can get a fix in for both of those in the next few days. Maybe exclude quay.io for now?

@sjhx Would probably wait for the fix.
As all our images are pulled from quay.io, I am not really sure how to exclude it.
Is there a possibility to mount the quay.io credentials in portieris somehow during admission?
Meanwhile below are the portieris logs that I see after another update.
I0624 15:11:44.568789       1 controller.go:65] Processing admission request for CREATE on
I0624 15:11:44.575251       1 controller.go:111] Getting policy for container image: quay.io/container/repo:1.1.1-1   namespace: misc
E0624 15:11:44.586447       1 secret.go:62] Error: secrets "quay.io" not found
E0624 15:11:44.587147       1 controller.go:174] secrets "quay.io" not found
I0624 15:11:44.587167       1 controller.go:191] policy.Simple {[]  }
E0624 15:11:44.587283       1 responder.go:87] trust: policy denied the request: Deny "quay.io/container/repo:1.1.1-1", no valid ImagePullSecret defined for quay.io/container/repo:1.1.1-1
I0624 15:11:44.587301       1 controller.go:150] Deny

@SrivigneshM did you resolve the "no valid ImagePullSecret defined for quay.io" issue mentioned above ?
I get the same, with a brief look I didn't find relevant information online except your issue.


Finish setting up allcontributors bot

https://allcontributors.org/docs/en/bot/installation

I think I got it, when we merge PRs from participants we should make sure to ping bot


Fix issue #6977 parse url failed when the host is IP

Fix issue #6977 . See https://github.com/golang/go/issues/19297

apologies, closing as duplicate of #8089


dep: replace anyhow with eyre in examples

Motivation
Slow move from anyhow to eyre throughout the crate
Solution
Make examples use eyre.
PR Checklist

[ ] Added Tests
[ ] Added Documentation
[ ] Updated the changelog

test failure is unrelated I think?


Compute grammar test coverage

If you want to know how much of your grammar is covered by test, we could overwrite or extend the document builder and count which rules and paths are used how often….
The benefit is when you go for reaching 100% coverage you might have tested enough and no testing gap. It also makes sense for transpiling one language into another. You have some progress measurement…

[ ] intercept document building and mark rules that are handled
[ ] print out gaps or percentage what is not covered after the test script
[ ] maybe a own command in the CLI like langium cover

Extras:

[ ] add a generator for missing test cases?

The challenge is that there's a second dimension to it - what you test about the grammar elements: type system, validation, completion etc


Added LinkedIn support

Thank you for your PR, but I have a better way and it is under development.


[Feature Request] Option to choose where to open MTE

I know that we can set hotkeys for different locations for the MTE (under, right, floating), but I'd like an option to set where to open it when clickiung on the icon in the left sidebar.
Also, the MTE commands are not available in the Command Palette (at least for me), they only works by hotkey.

Until I add settings pane, can you right click on the icon to open the menu and choose the direction where you want to open the editor?

Similar issue #24

Until I add settings pane, can you right click on the icon to open the menu and choose the direction where you want to open the editor?

Oh, good, thanks!

I have pushed some changes into 0.3.0. Can you please check and let me know if this works for you?
You can set this direction from the settings (default direction)

I have pushed some changes into 0.3.0. Can you please check and let me know if this works for you? You can set this direction from the settings (default direction)

it works, thanks! :)


How to set the window to be transparent but the internal components to be opaque ?

How to set the window to be transparent but the internal components to be opaque ?

You want this:
https://pub.dev/packages/flutter_acrylic


500 Error from AppService

I followed the instructions to deploy as an Azure Web App using Github Actions.  Everything deployed as expected, but when I open the web app url I get a 500 error.  The only way I can get in is to turn off authentication.   Am I missing something?  I'm pretty new to Azure so any assistance would be appreciated.

Hi @tirrellcotton,
Thank you for your feedback. Can you elaborate on the actual 500 error? Enabling the authentication is a highly recommended action so only authorized users can access it. Also, can you let me know the version of the tool you are running?
Thank you.
-Bryan

If you could provide any logs/error details, I can try and research what would be causing the issue. Here are some guides to help you research/find error logs.
https://learn.microsoft.com/en-us/azure/app-service/overview-diagnostics
https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-http-502-http-503
https://learn.microsoft.com/en-us/azure/app-service/troubleshoot-diagnostic-logs#enable-application-logging-linuxcontainer
-Bryan

Don't know why I cannot seem to get any logs, but do have a question.  When adding the app service via GitHub actions, step 3: Enable Azure Web App Authentication I selected "Microsoft", but what name should I have entered?  It seems my issue is with authentication and I feel this is where things may have gone wrong.  I entered the same name as my app service, was this correct?

From a code perspective, there is very little interaction with Authentication, Primarily, the tool just looks for specific headers and if they are present, it enables some functionality. A 500 Error indicates the request got to the server, however, it threw an exception.
If you could provide any screen shots, if would help:

The 500 error
The Developer Tools console when the error occurs (f12 in your browser)
Then Authentication configuration of your app service (Be sure to blur any sensitive values)

-Bryan

Here you go, thanks.

OK, thank you for the info. I think seeing the actual 500 error being thrown is critical, as it will specify what the issue is. I would recommend using the links above to investigate the App Service diagnostics. This will be different if you are using a Windows vs Linux App Service.
You may want to create a new app service and try to the process again. Perhaps something got messed up in the intial deployment.
-Bryan

Redeployed as a Windows app service instead of Linux and that seems to work.  Thanks for your time 😊

Thank you for the update. I have my dev site deployed to a Linux app, so perhaps something just got out of sync on your initial deployment. Glad to hear it's working for you now!

Bryan


Friction calculation not correct

When objects collide with each other or the border the friction calculation is incorrect in some way.
When for example an circle is colliding with the border it gets resolved in a way that the circle goes in the direction it came from together with a lot of angular velocity.

It was caused by a >= which should have been a <=


How to use EarlyStopping and ModelCheckpoint

The logs passed into the on_epoch_end method of both ModelCheckpoint and EarlyStopping seems to have the loss field set to a string, which leads to exception when comparing with the last best result. Following codes in trainer.py,
 407     @property
 408     def __training_tqdm_dict(self):
 409         tqdm_dict = {
 410             'loss': '{0:.3f}'.format(self.avg_loss),
 411             'epoch': '{}'.format(self.current_epoch),
 412             'batch_nb': '{}'.format(self.batch_nb),
 413         }

How should I use the ModelCheckpoint and EarlyStopping, is adding my own field in progress field when returning from training_step the way to go? What is the purpose of converting to string?
What's your environment?

conda version (no venv)   4.7.12
PyTorch version   1.2.0
Lightning version   0.5.0
Test-tube version  0.7.1

tqdm_dict is strictly for logging. To use early stopping or checkpointing:

tell the checkpoint what you want to monitor (ie: your_acc_or_something).
In training_step return something like:

return {'loss':some_loss, progress:{'your_acc_or_something': someValue}}


Attempt to improve the PDFDocument error message for empty files (issue 5887)

Given that the error in question is surfaced on the API-side, this patch makes the following changes:

Updates the wording such that it'll hopefully be slightly easier for users to understand.
Changes the plain Error to an InvalidPDFException instead, since that should work better with the existing Error handling.
Adds a unit-test which loads an empty PDF document (and also improves a pre-existing InvalidPDFException message and its test-case).

Fixes #5887

/botio test

That should indeed be a more clear message. Thanks, also for adding a unit test!


whereColumn feature missing

Hi!
I think is not possible to use the whereColumn, right?
Is there any special reason because it is not implemented?
I will be happy to help :)

any update about this?

Not from my side.
I'm afraid I'm not using Mongo right now, so I don't have plans on working on this.

Adding a use case for whereColumn.
We recalculate items every day that is within the period (date_first_in_period, date_last_in_period).
        $builder = BenchmarkCacheModel::query()
            ->where('filter_alias', $filterAlias)
            ->whereColumn('date_last_in_period', '>=', 'updated_at');

        $keyItemArray = $builder->get()->toArray();

Unfortunately this does not work since whereColumn is not implemented: "BadMethodCallException: This method is not supported by MongoDB"
MongoDb itself can do this with aggregations. I will have to solve this for now with a raw call.

Tracked with PHPORM-138.

This would be possible only if we convert Query Builder to Aggregation Builder. There is some limitation on the query operators that can be used, that's why we didn't implement it in #2738.
The solution would be to write an aggregation like mentioned by @peterlembke in the previous comment.


Community events to map

Fixes Issue
fixes #4789

Changes proposed
First of all there are many changes I am proposing to successfully solve this issue.

Issue #4132 with  Pull request #4438 was closed by adding 4 location parameters(road, city, state, country) but to locate the event into Map we need 2 more parameters which is Latitude and Longitude so I have added that 2 parameters in the events section.
For testing purposes I have added these 2 parameters in this event "data\TRohit20\events\2023-06-13-DevRelCon-London.json"
Also, the pull request I mentioned was incomplete because though he had added the location parameters in the event, he had not made that changes into the mongoose model and hence the location was not going into database.
I have made the necessary changes to store the location parameter into the mongoose model.
I have fetched the events from "pages/map.js" and further passed that events to "components/Map.js"
I have also added an event icon in the "public" folder to show in the map that there is an event. (see the screenshot added)
Done necessary changes in "events" documentation.

Check List (Check all the applicable boxes)

[x] My code follows the code style of this project.
[x] My change requires changes to the documentation.
[x] I have updated the documentation accordingly.
[ ] All new and existing tests passed.
[x] This PR does not contain plagiarized content.
[x] The title of my pull request is a short description of the requested changes.

Screenshots

This is how the location parameters look like (i have added "lat" and "lon" into this to locate this in map)

As soon as we click into the event shown on map we can see the details of the event.

Note to reviewers

We have to put a URL in the "iconURL" field and therefore putting React Component (react-icon) is not working there. Can you provide me a link to some image which you wish to use there ? We cannot put a React Component there.

Sorry for my slow reply, I will take a look today

Done with necessary changes... You may review it now

Maps had a refactor recently, so you have a conflicts, please can you resolve and also check there are not further improvements/changes you wish to make after that PR got merged https://github.com/EddieHubCommunity/LinkFree/pull/7653

Done with necessary changes

Should I resolve the conflicts now or should I wait ?

Should I resolve the conflicts now or should I wait ?

The forms PR is merged, so you can go ahead and resolve the merge conflict.

Sorry for the conflicts, any questions let us know

Conflicts are solved now.

Conflicts are solved now.

Items I am looking at: (I will keep updating this list as I work through the PR)

image for event (can we use an icon?)
map to have toggle for users/events (do events need tags also?)
refactor the way location is done to remove duplicate with user location

PR where I will make further changes https://github.com/EddieHubCommunity/LinkFree/pull/8093


Table : Autocomplete input error styling

Issue Summary

Update error message style for Auto complete on Table
Auto complete should fire an event  valueError on error input values

Expected Result

Auto complete Error message style to be updated as per figma and should be displayed as tooltip outside the cell

Observed Result

The current autocomplete displays error message inside the cell

Acceptance Criteria

The Auto complete input should fire valueError Event when the error text prop is given.
The new styling should match the figma design

Decisions

Emit the value Error event on error user input and create a tooltip to handle error message.

Test Cases

not Applicable

@prashanth-offcl This issue is in PR review but has not been story-pointed. Moving to current sprint.