keras metrics discards imaginary part of complex input and casts to float by taking the real part when passing complex numbers to metric function

TF 2.4.1 - 2.7
When using a metric that accepts complex numbers, tf keras fit ends up discarding the imaginary part of the complex number and then casts the array to real when inputting y_pred and y_true to a metric function.
It works correctly when used as a loss function.  The casting only occurs when the loss function is used as a metric.
The error is because of casting done in metrics.py on line 609.  See the snippet below from this file.
  def update_state(self, y_true, y_pred, sample_weight=None):
    """Accumulates metric statistics.

    `y_true` and `y_pred` should have the same shape.

    Args:
      y_true: Ground truth values. shape = `[batch_size, d0, .. dN]`.
      y_pred: The predicted values. shape = `[batch_size, d0, .. dN]`.
      sample_weight: Optional `sample_weight` acts as a
        coefficient for the metric. If a scalar is provided, then the metric is
        simply scaled by the given value. If `sample_weight` is a tensor of size
        `[batch_size]`, then the metric for each sample of the batch is rescaled
        by the corresponding element in the `sample_weight` vector. If the shape
        of `sample_weight` is `[batch_size, d0, .. dN-1]` (or can be broadcasted
        to this shape), then each metric element of `y_pred` is scaled by the
        corresponding value of `sample_weight`. (Note on `dN-1`: all metric
        functions reduce by 1 dimension, usually the last axis (-1)).

    Returns:
      Update op.
    """
    y_true = math_ops.cast(y_true, self._dtype)  # THIS IS WRONG!
    y_pred = math_ops.cast(y_pred, self._dtype)
    [y_true, y_pred], sample_weight = \
        metrics_utils.ragged_assert_compatible_and_get_flat_values(
            [y_true, y_pred], sample_weight)
    y_pred, y_true = losses_utils.squeeze_or_expand_dimensions(
        y_pred, y_true)

    ag_fn = autograph.tf_convert(self._fn, ag_ctx.control_status_ctx())
    matches = ag_fn(y_true, y_pred, **self._fn_kwargs)
    return super(MeanMetricWrapper, self).update_state(
        matches, sample_weight=sample_weight)

Setting a break point at the start of this function, we see the following vars in debug:
y_true
<tf.Tensor 'IteratorGetNext:3' shape=(None, 256, 256, 1) dtype=complex64>
self._dtype
'float32'
y_true should remain of type complex64 but instead is casted to float32 by ignoring the imaginary part and keeping the real part.
A correct solution will make an additional check for complex data in addition to the system floatx() type and cast accordingly (i.e. if floatx() is float32, complex128 becomes complex64; float64 goes to float32).
I searched for this pattern,  tf.cast(y_true, self._dtype), and it occurs quite a bit throughout this file.  From what I can tell, all cases are incorrect and will cast complex data to float data and discard the imaginary part.  I also confirmed this bug exists in tf 2.7, reference https://github.com/keras-team/keras/blob/v2.7.0/keras/metrics.py#L1096-L1141

@isaacgerg,
Can you share a simple standalone code to reproduce the issue ? Thanks!

@sanatmpa1 Unfortunately no.  Its self evident from the code i shared that complex numbers get casted to float and the imaginary part is discarded.

@isaacgerg if you explicitly set metric's dtype to a complex type, it will use provided dtype for metic calculation as expected: for example, use tf.keras.metrics.MeanSquaredError(dtype=tf.complex64) instead of 'mse' (automatic dtype casts could be helpful in this particular case, but made the framework more, er, complex. So probably it's not a bug).
Does this solution work for you?
Example:
import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense

model = tf.keras.Sequential([Dense(32, 'relu'), Dense(1, 'sigmoid')])
model.compile(
    'sgd',
    loss=tf.keras.losses.MeanSquaredError(),
    metrics=[tf.keras.metrics.MeanSquaredError(dtype=tf.complex64)]
)

res = model.evaluate(
    np.random.normal(0, 1, [32, 64]),
    np.random.choice([0, 1], 32),
)

Out:
1/1 [==============================] - 0s 212ms/step - loss: 0.3171 - mean_squared_error: 0.3171
/usr/local/lib/python3.7/dist-packages/keras/utils/generic_utils.py:960: ComplexWarning: Casting complex values to real discards the imaginary part
  info += ' %.4f' % avg

print(res)
Out:
[0.31706303358078003, (0.31706300377845764+0j)]   # or something like this... the loss value is real and the metic value is complex
# Inputs here are real, so the metric value on the real axis. It's just a demonstation

Complex dtypes are not allowed for some metrics - binary CE etc., and framework will complain about it. Also you cannot simply use standard losses with complex numbers, because they are not ordered, and the loss should 'define' some kind of order. That said, you probably would build custom loss and metric. That case you could set default value of subclassed metric to the complex dtype.

@mishc9 I have a workaround but what you suggest will not generalize to my simple usecase. My metric essentially takes the normalmize(log(abs(complex_image))) of y_true and y_pred and then computes ssim on them.  Futhermore, as I show in the code, the metric framework ignores the complex datatype for general metric functions.  It works for yours because it is a special case.

It looks like the workaround provided should be sufficient, feel free to re-open if the workaround provided does not solve your use case.

@LukeWood this is clearly a bug that you should fix.

This is closed but is clearly a bug!

@isaacgerg if you explicitly set metric's dtype to a complex type, it will use provided dtype for metic calculation as expected: for example, use tf.keras.metrics.MeanSquaredError(dtype=tf.complex64) instead of 'mse' (automatic dtype casts could be helpful in this particular case, but made the framework more, er, complex. So probably it's not a bug). Does this solution work for you?
Example:
import tensorflow as tf
import numpy as np
from tensorflow.keras.layers import Dense

model = tf.keras.Sequential([Dense(32, 'relu'), Dense(1, 'sigmoid')])
model.compile(
    'sgd',
    loss=tf.keras.losses.MeanSquaredError(),
    metrics=[tf.keras.metrics.MeanSquaredError(dtype=tf.complex64)]
)

res = model.evaluate(
    np.random.normal(0, 1, [32, 64]),
    np.random.choice([0, 1], 32),
)

Out:
1/1 [==============================] - 0s 212ms/step - loss: 0.3171 - mean_squared_error: 0.3171
/usr/local/lib/python3.7/dist-packages/keras/utils/generic_utils.py:960: ComplexWarning: Casting complex values to real discards the imaginary part
  info += ' %.4f' % avg

print(res)
Out:
[0.31706303358078003, (0.31706300377845764+0j)]   # or something like this... the loss value is real and the metic value is complex
# Inputs here are real, so the metric value on the real axis. It's just a demonstation

Complex dtypes are not allowed for some metrics - binary CE etc., and framework will complain about it. Also you cannot simply use standard losses with complex numbers, because they are not ordered, and the loss should 'define' some kind of order. That said, you probably would build custom loss and metric. That case you could set default value of subclassed metric to the complex dtype.

@isaacgerg does explicitly setting dtype work?

@LukeWood I am not sure.  I ended up rewriting the code to not have to depend on a user specified "type." Much cleaner and more mathematically correct in my opnion.

@LukeWood I am not sure. I ended up rewriting the relevant keras code to not have to depend on a user specified "type" but use the given type. Much cleaner and more mathematically correct in my opnion.

Understandable.   I wonder if as a framework TensorFlow should raise a warning or error when a user casts a complex # to a real number.  Perhaps we should require an explicit call to tf.real()

I can't comment on whether or not the bug belongs to tensorflow or keras.  I am simply a user reporting the bug (i think its my duty when using OSS). But, my team fixed it in our own in-house version of keras.  We never had to modify tensorflow.
The issue is that math_ops.cast() inside of keras does not properly handle complex types.
For reference (since I have been using numpy for about 20 years):

np.array(np.complex(1,1)).astype('float32')
x-wingide-python-shell://124747328/7:1: ComplexWarning: Casting complex values to real discards the imaginary part
array(1., dtype=float32)

My expectation (which I am surely not alone in this) is that there would be a similar warning in tensorflow and if there isn't, keras would be aware that inputs could be complex and act mathematically correct.
Keras provides own casting library called math_ops as I am sure you are aware.
This library should handle complex data types correctly especially since they provide their own library on top of native tensorflow operations.
I know that many keras user likely don't have an idea why images would even be complex or assume if it is, that its complex throughout the network.  However, for many instances of remote sensing (my research area), this is not the case. I see you are Peter Gerstoff (sp?) student so your group should be quite familiar with these scenarios.
If I were at google, I would write unit tests to flush all of this starting with the behavior that complex types:

Issue a warning when they are truncated mathematically inconsistent as in the case of what numpy does for casting.  Note that np.real() and np.imag() give no such warnings -- it would make sense.
Correctly handle the management of complex data types since they are so abundant in real world data. This includes ensuring all the interfaces do not assume that data is real or if they do, they give an assertion.

Hope this helps.  Unfortunately, I cannot post any code due to restrictions.


Update lionweb-java

Should also get us the latest json format.

We're already using the new json format, but some classes still have the old name (https://github.com/LIonWeb-org/lioncore-java/issues/78, https://github.com/LIonWeb-org/lioncore-java/issues/77, https://github.com/LIonWeb-org/lioncore-java/issues/76)

I just released lioncore-java 0.0.24, so that the classes have the right names

I just released lioncore-java 0.0.24, so that the classes have the right names

Thanks a lot


Adds scanning in workflow and Makefile for inclusive terminologies

Signed-off-by: Rashed Kamal<EMAIL_ADDRESS>Pull request

What this PR does / why we need it

Adds scanning in workflow and Makefile for inclusive terminologies.
Updated go.worksum and few generated files

Which issue(s) this PR fixes

Fixes #240
Describe testing done for PR

Build tool woke scans to identify non-inclusive terminology. Use make scan-terms for local scanning. In CI, scanning is enabled via woke actions. Rules are provided by VMware
Additional information or special notes for your reviewer

Codecov Report

Merging #241 (7288d41) into main (39f643c) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #241   +/-   ##
=======================================
  Coverage   69.88%   69.88%
=======================================
  Files          17       17
  Lines         943      943
=======================================
  Hits          659      659
  Misses        263      263
  Partials       21       21

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Thanks for updating the go.work.sum file. Changes LGTM :shipit:


Effect of packed and align on representation

Discussion topic for the effect of #[repr(packed)] and #[repr(align)] on memory layout.
Both of these attributes have RFCs. We should document what current behavior is, what gotchas to watch out for (e.g., &x.foo where foo is a field of a packed struct may not be aligned), and what we can guarantee going forward.

&x.foo where foo is a field of a packed struct may not be aligned

To note for the future validity discussion: This creates an invalid reference, which briefly exists even if we cast to raw pointer immediately. How to handle that is not part of the current discussion area, but we should remember adding it for the next :)

A bit...subtle perhaps.

I worry about how would we teach this, and whether we can lint against people splitting this into two operations (e.g. let a = &x; let b = a as *const T;) where depending on many things, the split operation could be UB while the "joint" operation is not.
I still think that this is the best solution we've got, and that I don't think it is worth it to add a new sigil for it. But if we introduce this subtlety doing the wrong thing should ideally produce a hard error (e.g. can't do &x.foo on a packed struct, but &x.foo as *const T is ok - maybe similar for uninitialized memory).

#[repr(align = "n")] doesn't support under-alignment (packed must be used for that), so I don't think it has any effect on memory layout on its own. As far as I can tell from its RFC, its only defined effect is to increase the alignment of the type. This necessitates more padding, but in theory the compiler could also move fields around.
I cannot think of a good reason for the compiler to do so. The possibility of trying to align individual fields at some alignment higher than their natural alignment but lower than the struct's alignment seems extremely esoteric. That said, I do not know in what circumstances PGO might actually make this valuable.
The one thing I do think that we should guarantee is that a newtype with repr(align) lays out its field at index 0, so that you can safely cast a reference to an overaligned newtype to a reference to its field.
As for #[repr(packed)], the meaning is specified from this RFC. It is quite clear from this text that the primary effect on layout is simply in that it leads the compiler to consider fields to have lower layout requirements. With that in mind, I think that there is no other discussion of layout required here. If there are special cases for struct layout where the layout is different for different types of the same alignment, then those special cases should have to account for packing, rather than the other way round, I think.
For newtypes, a similar guarantee should hold as above: a packed newtype is represented exactly as the field type, except for its alignment. The effect of such a type is only to reduce alignment requirements.
In fact, I think that if we go with that definition, we could define packed's representation as follows:

If packed is applied to a newtype, its representation is identical except the alignment is lower.
If packed is applied to any struct, its representation is identical to a non-packed struct where each field were actually a newtype of itself with alignment lowered as in the first bullet.

@alercah

The one thing I do think that we should guarantee is that a newtype with repr(align) lays out its field at index 0, so that you can safely cast a reference to an overaligned newtype to a reference to its field.

It seems surprising to me that #[repr(align)] would have an effect here! I guess the reasoning is sort of "why else would you have put that repr there"? I think this may be an argument for guaranteeing that a struct with 1 field always places that field at offset 0, full stop.

That said, I feel like there is probably more work to be done than your comment might suggest. In particular, I think that the interaction with #[repr(C)] leads to stricter requirements here. It seems like we should be able to specify that #[repr(align)] has "the same effect as this clang annotation"  or what have you.

I meant that, theoretically, we could have #[repr(align(16))] struct WideU64(u64); laid out with the field at byte 9, putting the padding before the field.
But in general I think that's kinda silly and pointless.

https://github.com/rust-lang/rust/issues/54148

See https://oroboro.com/short-enum/, in particular its mention of the use of __attribute__ ((__packed__)) in GNU C to force the use of short enums on platforms where short enums aren't the default. See also https://github.com/rust-rfcs/unsafe-code-guidelines/issues/10#issuecomment-428018830.

I believe this is basically addressed by #31, so I've marked this as final-comment-period as well.

@briansmith I'm not entirely sure what points you meant to convey with those links, though I didn't read them yet. The purpose of this issue was to kind of describe how align/transparent work; are those links talking about details of things we should document? Perhaps take a look at #31, which has some (fairly short and not super detailed, admittedly) text on this topic, and make some suggested edits?


本地推理inference.exe打不开，提示AttributeError: 'list' object has no attribute 'keys'

Loading model cost 0.300 seconds.
DEBUG:jieba:Loading model cost 0.300 seconds.
Prefix dict has been built successfully.
DEBUG:jieba:Prefix dict has been built successfully.
INFO:root:emb_g.weight is not in the checkpoint
INFO:root:Loaded checkpoint './G_latest.pth' (iteration None)
Traceback (most recent call last):
File "inference.py", line 99, in
AttributeError: 'list' object has no attribute 'keys'
[12828] Failed to execute script 'inference' due to unhandled exception!

Windows11，通过conda安装了依赖环境，并且是在虚拟环境中启动inference.exe的。
inference/文件夹下的G_latest.pth和finetune_speaker.json是在https://github.com/Plachtaa/VITS-fast-fine-tuning/blob/main/LOCAL.md里面下载的（https://huggingface.co/datasets/Plachta/sampled_audio4ft/resolve/main/VITS-Chinese/config.json和https://huggingface.co/datasets/Plachta/sampled_audio4ft/resolve/main/VITS-Chinese/G_0.pth）
请问一下要怎么解决？

要改一下下载的json文件，speaker那一栏改成字典

要改一下下载的json文件，speaker那一栏改成字典

修改后为字典文件可以打开inference
finetune_speaker.zip


Implement game score as part of button

Reverse buttons to reflect game score within the button, and the home line between the buttons.

Fixed.


scons: Change to Kconfig build system

The PR contains the following changes:

Move all of the config options(env["CONF"]) from SConsopt to Kconfig files
Update build_opts files to Kconfig option formats
The Ruby Protocol files are only built if RUBY=y
Remove the default-default build target
Kconfig commands are included in the PR:

defconfig
setconfig
meunconfig
guiconfig
listnewconfig
savedefconfig
oldconfig
olddefconfig

Add the python3-tk package dependencies

Jira issue: https://gem5.atlassian.net/browse/GEM5-1211

Previous patch discussion: https://gem5-review.googlesource.com/c/public/gem5/+/56590
Jira issue: https://gem5.atlassian.net/browse/GEM5-1211

I pinged all maintainers on this as it's a major change to the way gem5 is configured at build time and it's a major change to how users build gem5. We should all take a close look at this "API" breaking change to make sure that it's not going to affect the gem5 users' workflows.
Below is an excerpt of a conversation that Roger and I had over email for other developers to see. (Bracketed text is mine.) This includes some answers to how this is changing gem5's build API.

For users:
How do I build gem5?
All of the available variants are listed in the “defconfig” directory instead of in “build_opts”.  You need to use the new build scheme to build gem5. Please follow the steps below.
Example:
Initialize gem5 build
[NOTE: THIS IS A MAJOR CHANGE]
$ scons defconfig gem5_build defconfig/<variant>

The command creates a new gem5.build directory in the build directory you specified, the gem5.build includes the Kconfig setting and global config headers.
Build gem5 instance
[NOTE: THIS IS A MAJOR CHANGE]
$ scons gem5_build/gem5.opt

The command builds the gem5 instance based on the Kconfig in the gem5.build.
We also support old build schemes for backward compatibility. The following commands are equivalent.
Old build schemes
$ scons build/<variant>/gem5.opt

New build schemes
$ scons defconfig build/<variant> defconfig/<variant>
$ scons build/<variant>/gem5.opt

How do I customize the gem5 build?
[NOTE: THIS IS A MAJOR CHANGE]
Users can use scons setconfig to change the config from the command line or use scons menuconfig to change the config with UI. Here's an example.
If users want to enable kvm support based on ARM defconfig, we can do the following commands.
setconfig:
$ scons setconfig gem5_build USE_KVM=y
$ scons gem5_build/gem5.opt

menuconfig (will list all of available build options):
 $ scons menuconfig

How do I find out what gem5's build-time options are?
[NOTE: THIS IS A MAJOR CHANGE]
Users can run scons menuconfig to view available options. If the gem5 binary is built, they will show the variables in buildEnv in ${BUILDDIR}/python/m5/defines.py
For developers:
How do I extend the gem5 build with new build-time options?
After configuring the options with KConfig, the build options would be available in env['CONF'] in Scons scripts. If the user want to build some files only if options enabled, they can add following code in the SConscript:
If env[‘CONF’][<option>]:
env.TagImplies(<tag for option>, ‘gem5 lib’)

Where  is the option name in the KConfig files,  is the tag option related to the  from KConfig files which is used to determine the file be compiled,  name can be different with .
If the source file want to build only if specified option enabled, write like:
Source(‘<any filename>[.cc](http://foo.cc/)’, tags=<tag for option>)

Where  can be anything you want.
How do I modify gem5's build-time options (e.g., remove an option)?
There are two way to define the build-time options
Configurable options: defined by KConfig
Global options: defined by Sconsopt
Developers should reach the correct files depending on the options they want to modify.
After modification and reconfiguring by scons defconfig, the build options will be in the env[‘CONF’] for scons scripts as mentioned before..   Here's an example to add an option in KConfig.
FOO
<type> <description of FOO>
Default xxx

For more information about how to add options to KConfig, please refer to the document: https://www.kernel.org/doc/html/latest/kbuild/kconfig-language.html

CC: @wmin0, @EarlOu

I think everything you say makes sense, Andreas. One thing I'll point out is that all of the benefits you list require that we do a lot of new education for our users. For instance, if this was committed as is, there would be no way for users to discover that scons menuconfig existed if they continue to use scons build/<variant>/gem5.opt. To achieve the benefits that you enumerate is going to require a lot of documentation updates and a lot of changes to the tutorials that we run.
As I like to say when I review papers, "the impact of this change would be much larger if" it also included updates to educate gem5's users. Without including these education updates (e.g., a blog post, documentation on the website, updates to tutorial information, etc.), the positive benefits that Andreas listed are unlikely to be realized by most gem5 users.

Hi Jason and all,
Let me answer the two topics first, and then the specific questions later:

What's the benefit of this change?
a. The change cleans up the existing configuration method. In the original flow, some explicit configuration variables are sticky and some are not, some will be exported to c++ code and some will not. This causes ambiguity and sometimes the same command will generate different results depending on if it's the first time using (due to the sticky) or not.
  1. In the new flow, all explicit configuration variables will become sticky, or will be demoted to environment variables. As part of making sticky variables clear, all explicit configuration now requires 2 steps or editing a config file instead of using the command line. This process makes it clear that those are "sticky".
  2. All configuration variables will be exported. It will be up to gem5 source files to decide what variables they actually want, and if there is no dependency on them
  3. After the change, it will be clear that we have 3 different types of configs:
             i. Configuration option - persist (sticky), need to be written with editor, and affect build product
             ii. Env variable - not persist, set with command line, and affect build product
             iii. Command line option - not persist, set with command line, and affect build process

b. As part of the benefit from a), now a user can use scons menuconfig <build_folder> to see and edit all available options. In the past, there was no way to list all possible options.
c. The new flow allows a user to fine-grand define what to be included in the final result. For example, if we'd like to work on a SoC platform with two types of ISA CPU cores, the existing flow only has the "ALL" option. The new flow allows us to only pick the two ISAs we want. In our use case, due to the large number of IP models we have, we've found this to be useful to minimize the build time and build size.
d. Some options should only be set when the specified options are enabled. For example, PROTOCOL should be specified only if RUBY=y, otherwise PROTOCOL is always empty whatever the developers try to set PROTOCOL in the variant files. For another example, USE_ARM_FASTMODEL can be enabled if USE_ARM_ISA and USE_SYSTEMC are enabled, otherwise, USE_ARM_FASTMODEL is always disabled.

What's the suggested way for new users to build gem5?
As suggested from the previous review, we'd like to make it easy for most of the users. After some internal discussion, we'd like to see if the following documentation proposal is OK for the reviewers:

For typical users: use scons build/<variant>/gem5.opt to build
For advance users who want to change their default build options:
1. scons defconfig <build_folder> build_opts/<variant>
2. scons menuconfig <build_folder>
3. scons <build_folder>/gem5.opt

For the specific questions from Jason:

For the "backwards compatible" build scheme...
a. Please see above answers on the suggested way to build gem5.
b. We decide to move all the variants back to build_opts. So old developers can still find them.

Under "how do I extend gem5 build with new build-time options" you said "After configuring the options with KConfig..."
a. To add a new option, one can add the option in the Kconfig closer to the source, e.g. USE_X86_ISA is in src/arch/x86/Kconfig. And then use the option in the corresponding Sconscript, e.g. src/arch/x86/Sconscript as mentioned in the previous answer.
b. The Source action in Gem5 Scons has tags attribute which make developers can select a group of actions. By default Gem5 build with gem5 lib. In addition, developers can use TagImplies to associate two tag. So we can use the tags to implement optional build.

if env['CONF']['FOO']:
   env.TagImplies('foo', 'gem5 lib')
Source('foo.cc', tags='foo')

However we think using tag to implement the optional build is too complicated for normal users. We would like to suggest in another simpler way. We can just check if the config is enabled or just return in the SConscript.
if not env['CONF']['FOO']::
   Return()

Under "how do I modify gem5's build-time options..."
Most of the options are still in the scons files. For the configuration options originating from sticky_vars, such as USE_ARM_ISA, USE_RISCV_ISA, PROTOCOL, BUILD_GPU and so on. They will be moved to the Kconfig file from Sconsopts in the same directory. Developers can find such options in the Kconfig file in the same directory. The users can also use menuconfig command to know the detail about configuration options

Type `scons menuconfig  to enter the menu config UI

To show the configuration option name, press ‘c’ button

To know the details about options, press ‘?’ to the specified option. Here I take the USE_ARM_ISA as an example.

It includes where the configuration options come from. What the kconfig gets the options and so on.
For the guiconfig, it will open the new windows to list all of the configuration options

@powerjg

I think everything you say makes sense, Andreas. One thing I'll point out is that all of the benefits you list require that we do a lot of new education for our users. For instance, if this was committed as is, there would be no way for users to discover that scons menuconfig existed if they continue to use scons build/<variant>/gem5.opt. To achieve the benefits that you enumerate is going to require a lot of documentation updates and a lot of changes to the tutorials that we run.
As I like to say when I review papers, "the impact of this change would be much larger if" it also included updates to educate gem5's users. Without including these education updates (e.g., a blog post, documentation on the website, updates to tutorial information, etc.), the positive benefits that Andreas listed are unlikely to be realized by most gem5 users.

We will let typical user continue to use scons build/<variant>/gem5.opt to build gem5. I think it's not required to update the tutorial document. In the other hand, we can provide the guideline about Kconfig for advanced user.

I can see a lot of good questions from Jason about documentation, and also a lot of them should be answered by documentation instead of review comment (so the future users can get same information on a nice page).
So I'd like to propose we review in the following way:

Roger should upload another RP on the website repo (https://github.com/gem5/website) to document the Kconfig usage. And we review the documentation there.
In this PR, we review the design and implementation to see if everyone agree this is a good change to gem5.

In the document review, most of the question should be answered by new patch in the PR instead of review comment to make sure future users get the right information.

Thanks for all of the great information, Roger! Adding this documentation to the website will be a great way to alleviate many of my concerns. Apologies for the wall of text below. I am not really looking for any specific responses to questions, but I want to give you my perspective on this change and why I've been pushing back.
I think that your 1a benefit (that this change makes all variables "sticky") is something that greatly improves the maintainability of the gem5 build system and is a huge improvement. I am strongly in favor of this change! I'm a bit disappointed that it comes with the added complexity of KConfig, though. From my point of view, these two changes, reducing the complexity of the build-time options and KConfig, should be orthogonal. The only part of this change that I'm pushing back on is adding support for KConfig.
I'm not going to unilaterally block this change, but I want to point out two worries that I have with having to maintain this going forward:
(1) When helping people that have problems with gem5, it is going to become increasingly difficult to reproduce their errors if they use a highly customized gem5 build. Additionally, we may find that there are ways to configure gem5 with KConfig that (a) are never tested and (b) are broken/unsupported. This isn't a problem for expert users like you all, but we have many users that are not as expert as you at building and maintaining Linux-based applications. I am strongly against saying "we won't answer questions on gem5 if you use a customized build." Instead, as maintainers, I would like to see us make it easier to provide help to users.
(2) If we ever want to move away from scons (which I think we do!), we are now going to have to find a build system that supports KConfig. I see that zephyr seems to supports cmake+KConfig through their own customization, but their build flow is incredibly complicated! (See this link https://docs.zephyrproject.org/latest/_images/build-config-phase.svg). I worry that building our own new customization to use KConfig on top of some other build system is going to be a large and costly undertaking which will make it more difficult to ever move away from scons.
Having worked with gem5 for the past 12 years and been one of the lead maintainers for the past 6 years, I have found the most difficult parts of gem5 to maintain is when we have non-standard implementations and non-standard use cases for things. SCons is one of the most difficult parts of the project to maintain, and I am afraid that this added customization is going to make it even more difficult to maintain going forward. Furthermore, the way that we have customized things makes it very difficult to find answers on Google when something goes wrong, which further increases the burden on the community to help users and makes maintenance more difficult.

As a side note and a potential data point that may help you understand where I'm coming from... I am very familiar with how to use KConfig. I've been compiling my own kernels for the past 20 years. So for me, I would have no problem using this new system, and I totally see it's benefits for people who are highly familiar with how to build and deploy Linux applications.
However, 90% of the students that I work with in my group, 90% of the students that attend the gem5 tutorials, 90% of the students that attended the gem5 bootcamp last year... they had never compiled a kernel and had never used KConfig. In fact, for many gem5 users, building gem5 is one of the first times they have ever built an application on the command line! By adding this as another dependency to use gem5, we are raising the barrier to entry and essentially gatekeeping people coming into computer architecture research. My personal goal (as a professor and as one of the leaders of this community) is to make computer architecture research easier and to bring more people into the community.
I hear you when you say "well, you can just build gem5 the old way, still." However, this isn't the whole problem, as I've pointed out above. The community is going to have to support users in using KConfig and support cases where things go wrong. This change is also going to make it more difficult to move to a better-known build system than our highly-customized scons implementation.
In general, I'm going to push back when I see changes that are increasing the difficulty in getting started with gem5. I know this isn't everyone's goals with this project, but I wanted to give you some of my perspective.

Hi Jason,
Thanks for explaining the concern. I would like to try to answer them.

it is going to become increasingly difficult to reproduce their errors if they use a highly customized gem5 build. Additionally

Actually current SCons has supported customized build already. For example you can enable the SystemC integration by the command below.
$ python3 /usr/bin/scons -C gem5 USE_SYSTEMC=1 build/ARM/gem5.opt

If you dig into the build directory build/ARM/gem5.build, there's no record about how you built. build/ARM/gem5.build/variable only contains the part in build_opts. It's the current difficulty to reproduce the build.
With KConfig we put all variables in build/ARM/gem5.build/config. People can just provide the config file to us to reproduce. I would say it's a cleanup of the way defining build variables. I think it would make maintainers providing help to users easier.

I worry that building our own new customization to use KConfig on top of some other build system is going to be a large and costly undertaking which will make it more difficult to ever move away from scons.

The issue we met today is because SCons doesn't native support such complicated build scheme. Therefore, we did lots of works in SConstruct and site_scons. To move away from Scons, we need to extract the works we did from current SCons. Previously we have already moved the codegen part into individual python tools. This time we want to extract the build variables from SCons. Moving configuration step out of SCons doesn't increase the difficulty of moving away from SCons. Instead, it provides the opportunity to share the config between different build system. We can develop a parallel build system with the same config and then get away from SCons.
I think zephyr is not a very good example because it contains too many different things to build in the build flow. In gem5 case, device tree is apart from the main build process dt.
Finally, I would like to mention again that the customized build is already enabling in current SCons but seems no document talking about that. The KConfig feature is just bring that into users' eyes and making it more easily to use. We can also hide the capability but I don't think you want going that way.
I'm not sure if I answer parts of your concern. Please tell me any thought :)
Thanks.

If you dig into the build directory build/ARM/gem5.build, there's no record about how you built. build/ARM/gem5.build/variable only contains the part in build_opts. It's the current difficulty to reproduce the build.

I strongly agree that this is a problem that should be solved! Fixing this is a big improvement to gem5.

The issue we met today is because SCons doesn't native support such complicated build scheme. Therefore, we did lots of works in SConstruct and site_scons. To move away from Scons, we need to extract the works we did from current SCons. Previously we have already moved the codegen part into individual python tools. This time we want to extract the build variables from SCons. Moving configuration step out of SCons doesn't increase the difficulty of moving away from SCons. Instead, it provides the opportunity to share the config between different build system. We can develop a parallel build system with the same config and then get away from SCons.

This makes a lot of sense to me. Thanks for the explanation.

Finally, I would like to mention again that the customized build is already enabling in current SCons but seems no document talking about that. The KConfig feature is just bring that into users' eyes and making it more easily to use. We can also hide the capability but I don't think you want going that way.

"Making it more easy to use" is in the eye of the beholder :). As people who have significant experience with KConfig, we agree that it's pretty easy to use. But for people who have never seen it before, a flat file is much easier to understand.
I am starting to come around a bit more to this 😄. Especially if we have some nice documentation (like an extension of what you wrote in the response above), I think that telling users "continue to build gem5 with build/<default build_opts>/gem5.opt and you can customize that build by running scons menuconfig build/<default build_opts> make sense.
I would suggest that we try to keep the "default" and the "expert" builds as similar as possible when describing them in documentation. I.e., you can say "you can use any directory name for the build directory, but we will use build/<variant or default>/gem5.opt in all of our examples." Let's try not to confuse people by showing many different possible ways. I abide by the Python mantra of "there is one way to do it", PEP 20 line 13.

I'd like to raise the potential issue that this may break some of our testing infrastructure. The tests run via "tests/main.py", which I call "testlib", will build the gem5 binaries they need to run the tests and I think some of the changes outlined here might break this.
The code for creating a gem5 binary for a test is here: https://github.com/gem5/gem5/blob/af72b9ba580546ac12ce05bfaac3fd53fa8699f4/tests/gem5/fixture.py#L186. As you can see it constructs the build command by utilizing the PROTOCOL variable and depeneds on the files "build_opts".
While most of our tests build a pretty vanilla "build/ALL/gem5.opt" binary to run tests, other target specific ISAs with specific protocols.
I think ./main.py list --length=long gem5/riscv-boot-tests should sanity test any changes you have. Tests invoked with that command try to "build/NULL_MI_example/gem5.opt".

Hi Bobby,
Thanks for raising that concern. I think fixing the issue is our responsibility for this PR. We will look at the issue and update the PR accordingly.
Hi Jason,

I would suggest that we try to keep the "default" and the "expert" builds as similar as possible when describing them in documentation. I.e., you can say "you can use any directory name for the build directory, but we will use build//gem5.opt in all of our examples."

Originally our plan is not telling people there's a way to have different build directory name in current document here and here to reduce the confusion. Instead we will add a whole new section under building and learning_gem5 part2 to tell about how to do customized build. We want only advanced users knowing that feature as needed.
But yes, if you think we should tell people there's a possibility. We can add your sentence to the current building document and redirect people to our new content if they want to know more detail. How do you think about that?
For the documentation, as mentioned above, we plan to support both general docs and learning gem5. We are working on a PR to website repo. Stay tune :)
Thanks.

Update tests script few minutes ago. Let it test if it can pass PR Github CI

Test: test-simpoints-se-restore-ALL-x86_64-opt Failed after test lib changes

Test: test-simpoints-se-restore-ALL-x86_64-opt Failed after test lib changes

It's not just this change that's failing this test. I think it's an issue with the test itself. We're looking into it.
The documentation updates that you suggest sound great to me as well!

Thanks @powerjg for help, now it can pass the GitHub CI.
I will start to write to document about kconfig usage, it may need to take about one and two weeks. So the PR will not update for a while. Once the document is finished, I will create another PR in gem5 website.
Feel free to ping me if you have any concern
Thanks

@gabemblack

From my side, I'm waiting for documentation on the gem5 website, then I'll provide a final review of this PR.

Hi,
We are drafting the document for the kconfig website. Once completed, I will update the PR.

gem5/website#79

@rogerchang23424 thanks for the updates. I will get to reviewing this ASAP. I was out the last week or so, and I'm still catching up. Hopefully, I can review within 1 week. Ping me if you don't hear anything :).
Note: This is blocked by #252

BTW, I don't know how/why it was merged, but the file KCONFIG.md should probably be updated. See https://gem5-review.googlesource.com/c/public/gem5/+/58634

From my point of view, there's two small things on the to do list before we merge this.

I've asked at least one person unfamiliar with the patch to try it out and see if they understand things based on the documentation added to the website.
We need to have a changeset that allows all of the tests to pass.

Since the PR changes the methods to specify config options, I have created the other PR https://github.com/gem5/gem5/pull/478 to ensure the daily tests can work well after kconfig changed.

Hi maintainers,

I'd like to raise the potential issue that this may break some of our testing infrastructure. The tests run via "tests/main.py", which I call "testlib", will build the gem5 binaries they need to run the tests and I think some of the changes outlined here might break this.
The code for creating a gem5 binary for a test is here:
https://github.com/gem5/gem5/blob/af72b9ba580546ac12ce05bfaac3fd53fa8699f4/tests/gem5/fixture.py#L186
. As you can see it constructs the build command by utilizing the PROTOCOL variable and depeneds on the files "build_opts".
While most of our tests build a pretty vanilla "build/ALL/gem5.opt" binary to run tests, other target specific ISAs with specific protocols.
I think ./main.py list --length=long gem5/riscv-boot-tests should sanity test any changes you have. Tests invoked with that command try to "build/NULL_MI_example/gem5.opt".

Hi maintainers,
To fix the issue of daily tests after the kconfig build system introduced, I have added some commits to fix daily tests and test in #478. The test works well as the action(https://github.com/gem5/gem5/actions/runs/6602383103) with the current develop branch. Cloud we start merge process of the PR?
Thanks

Could we start the merge process of the PR?

There's one more potential blocker and one other small bump:

What about the weekly tests and the GPU tests? I think the tests are fine, but I don't fully understand the new build system.
Bobby is on vacation this week. Unfortunately, he is the only person who knows how to merge changes to the GitHub Actions workflows. There is a complication as these files have to be on the stable branch instead of develop.

Could you look into (1) above and then ping Bobby in 1 week to merge?

What about the weekly tests and the GPU tests? I think the tests are fine, but I don't fully understand the new build system.

PR for weekly tests and gpu tests: https://github.com/gem5/gem5/pull/503

Could you look into (1) above and then ping Bobby in 1 week to merge?

Sure

@rogerchang23424, I want to talk about this during the gem5 developers meeting tomorrow. Could you let me know what the current state is and if there's any blockers?
BTW, I apologize for the bad time zone for you! We will try to schedule this at a better time in the future.

I want to talk about this during the gem5 developers meeting tomorrow. Could you let me know what the current state is and if there's any blockers?

I have draft the document for kconfig in gem5/website#79 and include the build steps change after the kconfig merged.
For the tests, I have opened the #478 and #503 for daily and weekly tests respectively. To fix the test issue, I have change the build steps of daily tests and replace the original Gem5Fixture with Gem5KconfigFixture and works well in my local machine. I think it will not break the tests after the PR merged.

BTW, I apologize for the bad time zone for you! We will try to schedule this at a better time in the future.

The meeting time is 1:00 am Nov. 10 in Taiwan time zone. I think schedule the meeting in the afternoon(Pacific standard time) will be better for me. It's morning in Taiwan time.

I'll give it until tomorrow to merge, but just to confirm @rogerchang23424 : Are you ok with everything?

I'll give it until tomorrow to merge, but just to confirm @rogerchang23424 : Are you ok with everything?

Sure

Thanks for merging the PR

Thanks Jason, Bobby, Roger and also Gabe! It's a long journey to get this in! Also thanks everyone who contributes to the discussion!


Net analyzers update to version 8.0.0

Update Microsoft.CodeAnalysis.NetAnalyzers and AnalysisLevel to version 8.0.0
Fix couple of issues newly identified

✅ No public API change.


Plugin hasn't reliably worked for me in a few weeks

I used this plugin for a few months with little issue, but as of a few weeks ago, when I drag and drop an image, the default Obsidian behavior occurs: it downloads it into the attachments folder and transcludes it.
I've tried disabling every other plugin, I've tried generating a new Client ID, I've even tried using the plugin in a new vault just to see how it works - and every time, the default Obsidian behavior occurs. I've messed with about every setting and nothing has worked.
I don't really know what else to report or say, since I'm not getting an error message or anything. The expected behavior isn't occurring, but nor does it seem to register this failure as a bug. I've tried looking at the console and found nothing.
On Windows 10, Obsidian v0.14.6 - the latest - and the plugin is up to date.

Windows 7 x64, Obsidian 1.4.16, Imgur plugin 2.4.0. Unfortunately, the plugin does not work on my end at all: either drag'n'drop or paste via clipboard produce a local image copy, not a remote copy on Imgur.

I've managed to make it work somehow, except for drag'n'drop from a browser (only from a disk). However, there is Emo for Obsidian that has drag'n'drop from a browser. Consider adding that feature to your plugin, @gavvvr.


Efficiency gain by having YASL_peekcstr() return the managed reference (possibly const)

The current implementation/API of YASL_peekcstr() looks at the object on top of the stack and if it is a string, copies it to a new buffer and returns the new buffer pointer. However, since the operation specifies that it is peeking the stack data, could we safely get a more performant API by returning the C-string pointer itself? Then callers of this function (such as YASL_popcstr()) will need to allocate a buffer for the data if they want it, which popcstr would, but callers that are only interested in a read-only peak of the string get a more performant function call.

peekcstr unfortunately cannot, since the string may not be nul-terminated (due to the way strings are implemented in YASL).
However, I've added YASL_peeknstr in v0.13.3 which behaves like this (returning the string itself instead of copying it.


TF World '19 Talk: Facemesh with TF.js in the browser

Just saw the wonderful Unlocking the power of ML for your JavaScript applications with TensorFlow.js talk, link: https://youtu.be/kKp7HLnPDxc.
I'd also like to try the code of using Facemesh in the browser. Following the example code in the video at 25:48, I went to https://cdn.jsdelivr.net/npm/@tensorflow-models/facemesh.
However, I get the message "Failed to resolve the requested file."
Will you please help me find the Facemesh model?

This issue is more suitable for TensorFlow TFjs repo. Please post it on TFjs repo from here.. Thanks!

@kavikode
Please, let us know if you have posted the issue in TensorFlow TFjs repo ?. Thanks!

I am closing the issue as this issue is related to TFjs repo. Thanks!


Improve async validators

Currently, all validators that are functions are pushed into angular's $asyncValidators api. Per this comment there can be issues with this.
I'm suggesting that you can optionally specify a function as synchronous if you want to avoid those issues. I think it'd work something like this:
vm.fields = [
  {
    // ...
    validators: { isFoo }
  }
];

function isFoo($viewValue, $modelValue, scope) {
  return $viewValue === 'foo';
}
isFoo.isSync = true;

side note: For a short while, angular-formly supported the opposite of this with an isAsync property but that was just cumbersome and confusing so we opted to get rid of it and make all function validators use the async pipeline. This feature would be an optimization you could make if some of your validators actually make API calls. However, I would be willing to bring that back if people think it's a better solution/api (which it very well may be).

What's the purpose of having a single validators object currently?  If it's to keep things simple, I think the simplicity is lost when needing to add new properties to disambiguate between them.  Using the existing angular syntax seems the simplest and easiest to understand, since there's nothing new to learn:
vm.fields = [{
  validators:{},
  asyncValidators:{},
}]

Thanks for commenting @a-laughlin. I actually like that idea a lot. Unless someone comes along with a good rebuttal, I think this is what I'll do.
This can be implemented as a "smooth" breaking change, as in, I'll add the asyncValidators property, and add a console warning to any non asyncValidators validator that returns a promise that this api is being deprecated and they must either return true or false, or they must use the new asyncValidators api.
Would love more input (even if it's a :+1:). I think this is where I'm going.

:+1:

:+1:

Released as 6.18.0. Not a breaking change. Simply allows you to add asyncValidators explicitly. Also, you can specify formlyConfig.extras.explicitAsync which will prevent any of your validators from being placed onto the $asyncValidators pipeline.

Just updated the example

Hi Kent,
I am using this async validator as shown in the example, except that I'm not using the loader animation so there is no timeout of 1sec.
I noticed that the error thrown shows up in the browser console as soon as i finish typing an invalid value, but neither the field is set as invalid nor the validation message shows up below the field immediately. I understand its because of the blur model option. How can I modify it to update the field too immediately?

Is there a way to run multiple async validators on a field?  For instance, I need to make one call to see if an access code is registered more than 5 times, and another call to see if the person is already in the database, and these are two separate calls to two separate Controllers.


Cannot reproduce the reported results

Hi,
Thanks for your great work.
I ran the code with ResNet-50 on RefCOCO and got the results 69.75/69.07/69.46 on val/test A/test B. The result on test A is much worse than the reported result 73.11.
Could you give some suggestions on this situation?
Many thanks.

Hi, I encountered a similar problem during replication of the training process, where the RefCOCO accuracy was observed to be around 69.5% (val). How did you solve it? Thanks!

Hi, the weights of our model have been made available at the following link: https://pan.baidu.com/s/1jaOJKdIg1t8wnWrxgCkkRA?pwd=vmyv
Please note that you may need to enter the password "vmyv" to access the files.

me too,i can't reproduce the report results with GPU 3090,especially in backbone res50

I've noticed that the experimental results may vary across different machines and environments. Initially, I conducted the experiments using a 40G A100 GPU with Python 3.9 or 3.10. You might be able to reproduce our results by training on a machine with similar specifications. In fact, we have also released weights on ViT and Resnet for reproducing the results.

me too,i can't reproduce the report results with GPU 3090,especially in backbone res50

I've noticed that the experimental results may vary across different machines and environments. Initially, I conducted the experiments using a 40G A100 GPU with Python 3.9 or 3.10. You might be able to reproduce our results by training on a machine with similar specifications.

Thanks, I'll try it again. Happy new year!

Thanks, I'll try it again. Happy new year!

Happy new year! You can also try our new project Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation, which has over 10 IoU improvement using adapters with fewer parameters. The project will be open-sourced in January 2025.


Some tests failing

codegenerationtest and actiontest are failing:
codegenerationtest: Decoder.incPos: buffer overflow
actiontest: access violation

Solved by 1556b5c7463a893ec13795e866cc5f05932024d0 & 402c695686865564f967a59d46866c557bbf49a4


Improve mobile guide navigation

Proposal
Screencap

Current Status
Screencap

Additional Changes

Sorry for delayed response! This is really nice, thanks — were you thinking that this would apply to all screens rather than just the guide? (the site isn't all that well architected to make these things straightforward... I mean to rewrite the whole thing as a Svelte app)

No worries! It just as well may — although if we were to extend it to the other routes I'd go for Apple's approach and adjust the guide's TOC accordingly, maybe showing the current position as well (a là Vue, which highlights the current section in the TOC).

That sounds excellent!

Well, Vue.js is relying on colour alone. I could have applied a custom User Stylesheet (for example due to a need for High Contrast) which overwrites it.
Best Practice is to use another hint like bold or a small icon to indicate current navigation position.

@flagello Thanks for updating me on cross-browser support :-)


Add ceil_sqrt(x) function, and its tests

Add the ceil_sqrt(x) function that takes in an integer and returns the square root of x, rounded up.
ceil_sqrt increments i from 0, and stops when the square of the i is no longer less than x.
As a result, it returns an integer.

This change is

LGTM

Thanks!


New Create/Edit Page should display blank page

Display a listing in Listing Management.
Using the triple lines, click on Submit a Listing
Page display the data from the previous listing.  It should display a blank page.

I'm closing this issue as it is a duplicate of #151


Resolves issue 35

This pull request is intended to resolve briancherne/jquery-hoverIntent#35, and improve convention and compression. Also, the track, compare, and delay functions have been scoped higher to avoid re-instantiation.
Here updates the fiddle from briancherne/jquery-hoverIntent#35 as a test for the improvements: http://jsfiddle.net/7px1e2kd/2/.

Thanks for the pull request! Unfortunately it creates a new problem when hoverIntent is invoked on a jQuery collection containing more than one element. See an update to your example here, which demonstrates the problem: http://jsfiddle.net/7px1e2kd/3/ .
Choosing where and how to store state can be a little tricky for this plugin, since much of it is per-instance-per-element data. The good news is I've already implemented a fix, which should land soon, so I can close this pull request. (There were also some unrelated features in the PR, but I did take some inspiration from your refactoring and did a little bit of cleanup there--thanks again!)


Radix Sort Article

Have you read the Contributing Guidelines on Pull Requests?
Yes
Description
I have added an article on Radic Sort in the resources folder.
Checklist

[ ] I've read the contribution guidelines.
[ ] I've checked the issue list before deciding what to submit.
[ ] I've edited the README.md and link to my code.

Related Issues or Pull Requests
#1541

LGTM

LGTM

@all-contributors please add @goelesha for code

@all-contributors please add @goelesha for code


feat(ui): band chart graphic

Adds the graphic in anticipation of the Band Chart view type

[ ] CHANGELOG.md updated with a link to the PR (not the Issue)
[x] Well-formatted commit messages
[x] Rebased/mergeable
[x] Tests pass
[ ] http/swagger.yml updated (if modified Go structs or API)
[ ] Documentation updated or issue created (provide link to issue/pr)
[ ] Signed CLA (if not already signed)

🔥

Related to #19341

We can close this one now. It is already covered in https://github.com/influxdata/influxdb/pull/19341


chore: add colors in tailwind.config

Add color variables to use in the design home page components.
Closes #7/colors

Please clarify  "rename all colors used in the code" ? That is a big work, that should be done by every team member.

@VOlekh how can we merge this without renaming the classes? This merge will break all the pages. It is not a big work. It is search+replace in files. It would take me 5 mins.

@VOlekh please always request all developers on team (@amanuela97 @harshbrahmxatriya) to review. It's important for everyone to know all the changes in the code base.

@VOlekh please always request all developers on team (@amanuela97 @harshbrahmxatriya) to review. It's important for everyone to know all the changes in the code base.


Is this necessary to register in telerik web site to set up telerik UI packages

Problem encountered on https://dotnet.microsoft.com/learn/aspnet/hello-world-tutorial/install
Operating System: windows
Provide details about the problem you're experiencing. Include your operating system version, exact error message, code sample, and anything else that is relevant.

I'm not sure @laxmanmahar. This would be a question for Telerik. Do you remember which web page took you to this tutorial?

I'm not sure @laxmanmahar. This would be a question for Telerik. Do you remember which web page took you to this tutorial?

Mam I took normal webform in asp.net, but when I setup my telerik package inside my visual studio it doesn't seem(rad gridview in toolbox) also have create problem, beacoz I don't have any idea of telerik control

Then certainly you don't need this tutorial to do that. But I am not sure how to solve your problem with the Telerik package. I'd ask a question on Stack Overflow or contact Telerik. This tutorial is for people wanting to learn ASP.NET Core.

Then certainly you don't need this tutorial to do that. But I am not sure how to solve your problem with the Telerik package. I'd ask a question on Stack Overflow or contact Telerik. This tutorial is for people wanting to learn ASP.NET Core.

Thank you mam, for the kind help, hope this will help to solve my problem.


⚖ Register Validator: 🔥STAVR🔥

Network Choice
dentrite-1
Who are you?
The STAVR team has been working on blockchain security and verification since 2020(in the crypto industry since 2017). We are trusted by over 5,400 delegates and securing more than 16M$ in user funds across  of projects based on blockchains such as Cosmos Solana Polkadot Etherium and several unique proprietary blockchains.
https://stavr-team.gitbook.io/
Delegator address
axone12rhavel5gnuefexre7ulk2l4vxug5tez5pv0p7
gentx
{"body":{"messages":[{"@type":"/cosmos.staking.v1beta1.MsgCreateValidator","description":{"moniker":"STAVR","identity":"F2F91999ECCC092F","website":"https://stavr-team.gitbook.io/","security_contact":"stavr.tech@gmail.com","details":"A team of professional and reliable validators.Safety first.Stake with us and profit with the mark of quality. Monitoring and Support 24\\7"},"commission":{"rate":"0.050000000000000000","max_rate":"0.200000000000000000","max_change_rate":"0.200000000000000000"},"min_self_delegation":"1","delegator_address":"","validator_address":"axonevaloper12rhavel5gnuefexre7ulk2l4vxug5tezy5qamu","pubkey":{"@type":"/cosmos.crypto.ed25519.PubKey","key":"NA9W9Croc8N+VCvljnk5EeeTainSrMtDKoDfAc607ig="},"value":{"denom":"uaxone","amount":"1000000000000"}}],"memo":"f7a8854f3be91d1262a90468b9f82709a3e2d9fa@<IP_ADDRESS>:26656","timeout_height":"0","extension_options":[],"non_critical_extension_options":[]},"auth_info":{"signer_infos":[{"public_key":{"@type":"/cosmos.crypto.secp256k1.PubKey","key":"AhKOqbJdTLnYsOGi04ng1zZ4a1ACaBD2qLn6KiV4il2O"},"mode_info":{"single":{"mode":"SIGN_MODE_DIRECT"}},"sequence":"0"}],"fee":{"amount":[],"gas_limit":"200000","payer":"","granter":""},"tip":null},"signatures":["BwG62+42WX4q4iYX+PCnsMhoEfnLl3OhAc+aPDXswPpCxWXi4jxD/TmjWQlKm4lcOzyCZPtAvHRqjOu+ImLodQ=="]}

😉 Here is the corresponding PR: https://github.com/axone-protocol/networks/pull/804

Done https://github.com/axone-protocol/networks/pull/804


[web.archive:youtube] Missing uploader and/or upload_date from some captures

Checklist

[X] I'm requesting a site-specific feature
[X] I've verified that I'm running yt-dlp version 2022.07.18 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[ ] I've read about sharing account credentials and I'm willing to share it if required

Region
United States
Example URLs
See description
Provide a description that is worded well enough to be understood
It is my understanding that the archive.org/Wayback machine extractor relies on captures of many different versions of YouTube webpages so the extractor has not covered all of the cases. Here are some URL I found which can be used to help improve the extractor not pulling metadata for some.
Did not extract upload_date and uploader:
https://web.archive.org/web/0/https://www.youtube.com/watch?v=8XeW5ilk-9Y
https://web.archive.org/web/0/https://www.youtube.com/watch?v=Q_yjX80U7Yc
https://web.archive.org/web/0/https://www.youtube.com/watch?v=c5uJgG05xUY
https://web.archive.org/web/0/https://www.youtube.com/watch?v=xWTLLl-dQaA
Did not extract uploader only:
https://web.archive.org/web/*/https://www.youtube.com/watch?v=9eO1aasHyTM
https://web.archive.org/web/20200416034815/https://www.youtube.com/watch?v=Cf7vS8jc7dY&gl=US&hl=en
https://web.archive.org/web/0/https://www.youtube.com/watch?v=Lg6b-yuabjI
https://web.archive.org/web/0/https://www.youtube.com/watch?v=KDxNEGq0x1U
https://web.archive.org/web/0/https://www.youtube.com/watch?v=NxAOOip1nZ4
https://web.archive.org/web/0/http://www.youtube.com/watch?v=mL-GP5qkyAM
https://web.archive.org/web/0/https://www.youtube.com/watch?v=rRCmytAVw1E
https://web.archive.org/web/0/https://www.youtube.com/watch?v=kbh4T_b4Ixw
https://web.archive.org/web/0/https://www.youtube.com/watch?v=cTf4oaro_es
https://web.archive.org/web/0/https://www.youtube.com/watch?v=D1wLshtOWsg
https://web.archive.org/web/0/https://www.youtube.com/watch?v=bGPM893vLog
https://web.archive.org/web/0/https://www.youtube.com/watch?v=_Bjua_wv4LY
https://web.archive.org/web/0/https://www.youtube.com/watch?v=kRusJD4Q1Ks
https://web.archive.org/web/0/https://www.youtube.com/watch?v=VO23JnCOW4s
https://web.archive.org/web/0/https://www.youtube.com/watch?v=d65VUt6eRgM
https://web.archive.org/web/0/https://www.youtube.com/watch?v=SOm7mPoPskU
https://web.archive.org/web/0/https://www.youtube.com/watch?v=qHL1aGEH4_4
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
[debug] Command-line config: ['-vU', 'https://web.archive.org/web/0/https://www.youtube.com/watch?v=Q_yjX80U7Yc', '--skip-download', '-j']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.07.18 [135f05e] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2022-06-16-git-5242ede48d-full_build-www.gyan.dev (setts), ffprobe 2022-06-16-git-5242ede48d-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.07.18, Current version: 2022.07.18
yt-dlp is up to date (2022.07.18)
[debug] [web.archive:youtube] Extracting URL: https://web.archive.org/web/0/https://www.youtube.com/watch?v=Q_yjX80U7Yc
[web.archive:youtube] Q_yjX80U7Yc: Fetching archived video file url
[web.archive:youtube] Q_yjX80U7Yc: Downloading CDX API JSON
[debug] [web.archive:youtube] Captures to try:<PHONE_NUMBER>0845,<PHONE_NUMBER>1719,<PHONE_NUMBER>0000,<PHONE_NUMBER>0000
[web.archive:youtube] Q_yjX80U7Yc: Downloading capture webpage
[web.archive:youtube] Q_yjX80U7Yc: Downloading capture webpage
[web.archive:youtube] Q_yjX80U7Yc: Downloading CDX API JSON
[debug] Default format spec: bestvideo*+bestaudio/best
[info] Q_yjX80U7Yc: Downloading 1 format(s): 34
{"id": "Q_yjX80U7Yc", "title": "Spray Paint Art by Clay Butler: Purple Fantasy Forest", "description": "Visit http://biturl.net/SprayPaintVideo to learn these techniques to create your own masterpieces in minutes!\n\nClay Butler painting a fantasy forest using only spray paint. Music from the Clay Butler Music production library. For more information on Clay's art or music, visit www.butlerproductionsmultimedia.com. Also, visit our blog at www.butlerproductionsblog.com.", "thumbnails": [{"url": "https://web.archive.org/web/20120414051423if_/http://i.ytimg.com/vi/Q_yjX80U7Yc/default.jpg?w=80&h=60&sigh=6og4ocFfS7zoz8R4qTrQbMiqvcQ", "filesize": 2057, "preference": 2057, "id": "0"}, {"url": "https://web.archive.org/web/20110126141816if_/http://i.ytimg.com/vi/Q_yjX80U7Yc/hqdefault.jpg?w=160&h=120&sigh=Jc-gD52z554qMcA9PwypmuiteaA", "filesize": 7739, "preference": 7739, "id": "1"}, {"url": "https://web.archive.org/web/20130119150131if_/http://i.ytimg.com/vi/Q_yjX80U7Yc/hqdefault.jpg?w=160&h=120&sigh=Jc-gD52z554qMcA9PwypmuiteaA", "filesize": 13890, "preference": 13890, "id": "2"}], "formats": [{"url": "https://web.archive.org/web/20121231194239oe_/http://r2---sn-nwj7kner.c.youtube.com/videoplayback?mv=m&gcr=us&signature=C5177EAD18A2994F169AE26D9DA48626A45AB448.24E37DBC6F20FAD43C6DC53CD6D8AA165429B182&ms=au&factor=1.25&id=43fca35fcd14ed87&fexp=927602,919358,916611,901467,920704,912806,928001,922403,922405,929901,913605,929104,913546,913556,908496,920201,913302,919009,911116,926403,901451,902556&source=youtube&sver=3&expire=1357006948&algorithm=throttle-factor&burst=40&sparams=algorithm,burst,cp,factor,gcr,id,ip,ipbits,itag,source,upn,expire&key=yt1&ip=<IP_ADDRESS>&upn=J_csn7yXBSQ&cp=U0hUTFBLVF9JUkNONF9IRVhKOmJFRUQ5enctYzZn&itag=34&ipbits=8&mt=1356982931", "filesize": 4691665, "ext": "flv", "width": 640, "height": 360, "acodec": "aac", "abr": 128, "vcodec": "h264", "format_id": "34", "format": "34 - 640x360", "protocol": "https", "resolution": "640x360", "dynamic_range": "SDR", "http_headers": {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-us,en;q=0.5", "Sec-Fetch-Mode": "navigate"}}], "webpage_url": "https://web.archive.org/web/0/https://www.youtube.com/watch?v=Q_yjX80U7Yc", "original_url": "https://web.archive.org/web/0/https://www.youtube.com/watch?v=Q_yjX80U7Yc", "webpage_url_basename": "watch", "webpage_url_domain": "web.archive.org", "extractor": "web.archive:youtube", "extractor_key": "YoutubeWebArchive", "playlist": null, "playlist_index": null, "thumbnail": "https://web.archive.org/web/20130119150131if_/http://i.ytimg.com/vi/Q_yjX80U7Yc/hqdefault.jpg?w=160&h=120&sigh=Jc-gD52z554qMcA9PwypmuiteaA", "display_id": "Q_yjX80U7Yc", "fulltitle": "Spray Paint Art by Clay Butler: Purple Fantasy Forest", "requested_subtitles": null, "_has_drm": null, "url": "https://web.archive.org/web/20121231194239oe_/http://r2---sn-nwj7kner.c.youtube.com/videoplayback?mv=m&gcr=us&signature=C5177EAD18A2994F169AE26D9DA48626A45AB448.24E37DBC6F20FAD43C6DC53CD6D8AA165429B182&ms=au&factor=1.25&id=43fca35fcd14ed87&fexp=927602,919358,916611,901467,920704,912806,928001,922403,922405,929901,913605,929104,913546,913556,908496,920201,913302,919009,911116,926403,901451,902556&source=youtube&sver=3&expire=1357006948&algorithm=throttle-factor&burst=40&sparams=algorithm,burst,cp,factor,gcr,id,ip,ipbits,itag,source,upn,expire&key=yt1&ip=<IP_ADDRESS>&upn=J_csn7yXBSQ&cp=U0hUTFBLVF9JUkNONF9IRVhKOmJFRUQ5enctYzZn&itag=34&ipbits=8&mt=1356982931", "filesize": 4691665, "ext": "flv", "width": 640, "height": 360, "acodec": "aac", "abr": 128, "vcodec": "h264", "format_id": "34", "format": "34 - 640x360", "protocol": "https", "resolution": "640x360", "dynamic_range": "SDR", "http_headers": {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36", "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", "Accept-Language": "en-us,en;q=0.5", "Sec-Fetch-Mode": "navigate"}, "epoch":<PHONE_NUMBER>, "_filename": "Spray Paint Art by Clay Butler - Purple Fantasy Forest [Q_yjX80U7Yc].flv", "filename": "Spray Paint Art by Clay Butler - Purple Fantasy Forest [Q_yjX80U7Yc].flv", "urls": "https://web.archive.org/web/20121231194239oe_/http://r2---sn-nwj7kner.c.youtube.com/videoplayback?mv=m&gcr=us&signature=C5177EAD18A2994F169AE26D9DA48626A45AB448.24E37DBC6F20FAD43C6DC53CD6D8AA165429B182&ms=au&factor=1.25&id=43fca35fcd14ed87&fexp=927602,919358,916611,901467,920704,912806,928001,922403,922405,929901,913605,929104,913546,913556,908496,920201,913302,919009,911116,926403,901451,902556&source=youtube&sver=3&expire=1357006948&algorithm=throttle-factor&burst=40&sparams=algorithm,burst,cp,factor,gcr,id,ip,ipbits,itag,source,upn,expire&key=yt1&ip=<IP_ADDRESS>&upn=J_csn7yXBSQ&cp=U0hUTFBLVF9JUkNONF9IRVhKOmJFRUQ5enctYzZn&itag=34&ipbits=8&mt=1356982931", "_type": "video"}

Yeah supporting every past version of YouTube is a difficult task, but I am open to continuously improving the extractor to cover more cases.

https://github.com/yt-dlp/yt-dlp/commit/19b4e59a1e1bf368078f90e7f735fa4576f97b64 fixes some of this, but I haven't checked with all the given examples


missing SecurityLib files

add to lib folder, or tell users that they need that library founded in your https://github.com/ircmaxell/SecurityLib/releases.
don't know if the installer install securitylib as well, but for users which like the manual download this can be frustrating.

You can see the full requirements of any composer project or library by looking in the composer.json.  For example: https://github.com/ircmaxell/RandomLib/blob/master/composer.json#L20-L23.  You should definitely look into learning composer even if you aren't using it since there are so many projects that do.

yes, I understand, but in the readme it's so simple to add a line stating "this project depends on xxx" since there's a sentence which describe how to install manually.

If I don't use composer, how do I add the SecurityLib properly? I get "Fatal error: Class 'SecurityLib\AbstractFactory' not found" from the Factory.php. I just want to try this library and not an PHP expert.

@Joohansson go to https://github.com/ircmaxell?tab=repositories and download the needed repository.


Upload code on Codecov

Closes #9

Codecov Report

:exclamation: No coverage uploaded for pull request base (develop@b1d2eb7). Click here to learn what that means.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             develop       #16   +/-   ##
===========================================
  Coverage           ?   100.00%
===========================================
  Files              ?         1
  Lines              ?        14
  Branches           ?         0
===========================================
  Hits               ?        14
  Misses             ?         0
  Partials           ?         0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b1d2eb7...d0bf285. Read the comment docs.


No route matches [GET] "/resource_pool/resource_pool/show_list"

In ResourcePool I clicked on ListView in toolbar and it crashed. It seems that there are some double things in URL that shouldn't be.

@miq-bot assign himdel


6704-readfile

Description
Just added ReadFileStdin class to mitmweb
Checklist

[x] I have updated tests where applicable.

afaik there are already tests for ReadFileStdin

[x] I have added an entry to the CHANGELOG.

Thanks.

fixes #6704


webauthn for signing

Is there an example in this repo for signing sha-3-512 hashes per these docs?
https://developers.yubico.com/WebAuthn/Concepts/Using_WebAuthn_for_Signing.html

@unicomp21 I'm not seeing any mention of SHA-3 in the FIDO2 specification, or any proposals to add SHA-3 based algorithms yet. Are you able to point to any specific documents where SHA-3 algorithms are used?

No there's no example for signing here, AFAIK there is no direct support for signing in the lib at all. Reading that document though, it shouldn't be too hard to implement.

I'll have to give it a try, thanks!
On Wed, Mar 6, 2024 at 12:07 PM Marlon Regenhardt @.***>
wrote:

No there's no example for signing here, AFAIK there is no direct support
for signing in the lib at all. Reading that document though, it shouldn't
be too hard to implement.
—
Reply to this email directly, view it on GitHub
https://github.com/passwordless-lib/fido2-net-lib/issues/478#issuecomment-1981496987,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEFL7M3PIWTUMJURPFQCLTYW5LMXAVCNFSM6AAAAABBGSJOBOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBRGQ4TMOJYG4
.
You are receiving this because you were mentioned.Message ID:
@.***>


ImportError: cannot import name import_by_path

I'm getting the foll error when running locally. Not sure of the issue any suggestions?
Running Sphinx v1.3.3
Exception occurred:
File "C:\Python27\lib\site-packages\readthedocs-1.0-py2.7.egg\readthedocs\privacy\loader.py", line 1, in
from django.utils.module_loading import import_by_path
ImportError: cannot import name import_by_path
The full traceback has been saved in c:\users\agumma~1\appdata\local\temp\sphinx-err-tg80oq.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at https://github.com/sphinx-doc/sphinx/issues. Thanks!

Its deprecated from Django 1.10!
https://github.com/djangonauts/django-pgjson/issues/45


fix: vue devtools setup

Oh, my bad, I deleted the forked repo and then the PR is auto closed 😅.  We can follow this PR to fix the https://github.com/vuejs/devtools-next/issues/236 issue. /cc @jbmaisner

Oh, my bad. I deleted the forked repo and then the PR is auto closed 😅. We can follow this PR to fix the vuejs/devtools-next#236 issue. /cc @jbmaisner

hi @webfansplz and thanks a lot for your help. I had no idea the bug's origin could be there ^^


Add transaction isolation level for Named blobs MySql DB

Set transaction isolation level to connections for Named Blob DB. The value is taken via configuration.

Codecov Report
Patch coverage has no change and project coverage change: -59.36 :warning:

Comparison is base (fd9943f) 73.27% compared to head (d77a23a) 13.91%.

Additional details and impacted files
@@              Coverage Diff              @@
##             master    #2496       +/-   ##
=============================================
- Coverage     73.27%   13.91%   -59.36%
+ Complexity    11089     2363     -8726
=============================================
  Files           783      789        +6
  Lines         62217    62957      +740
  Branches       7624     7697       +73
=============================================
- Hits          45591     8763    -36828
- Misses        14126    53682    +39556
+ Partials       2500      512     -1988

Impacted Files
Coverage Δ

...om/github/ambry/config/MySqlNamedBlobDbConfig.java
0.00% <0.00%> (-100.00%)
:arrow_down:

.../github/ambry/named/TransactionIsolationLevel.java
0.00% <0.00%> (ø)

...om/github/ambry/named/MySqlNamedBlobDbFactory.java
0.00% <0.00%> (-86.37%)
:arrow_down:

... and 632 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Fix VPC flowlogs and cloudrail plugins

Story details: https://app.shortcut.com/tenzir/story/38249


Does not automplete after colon

If I have the following scrips in my package.json:
scripts: {
dev:something: ",,.,,"
dev:another: "...."
}

When I type dev:s and press tab does not autocomplete dev:something

What version do you have? That should have been fixed in 0.3.0 https://github.com/dsifford/yarn-completion/commit/99ec65d7cad1e97a0b933a3c4801e68c862df0d3

For me it ends up repeating.
"scripts": {
  "test": "...",
  "test:client": "...",
  "test:server": "..."
}

Type te, tab, tab, tab

Then :, tab

It appends test to the already completed test:
I'm using Version: 0.5.0, Yarn 1.1.0

@cdeutsch Might be an issue in an old version of bash. What version are you running?
I'm not able to reproduce the issue on my machine.

That could be.
I'm using 3.2.57(1)-release (macOS Sierra 10.12.6)
Adding this to my .bash_profile fixed it for me
COMP_WORDBREAKS=${COMP_WORDBREAKS//:}

@cdeutsch Good to know. Thanks for posting your solution.
Also highly recommended:
$ brew install bash (and then follow the prompts to set as your login shell, e.g. $ chsh -s /path/to/bash4 yourusername)
Also unrelated but highly, highly recommended:
Replace those pesky BSD utils with their GNU counterparts: https://apple.stackexchange.com/a/69332

Same here, after adding COMP_WORDBREAKS=${COMP_WORDBREAKS//:}  seems to work.
$ echo $COMP_WORDBREAKS
"'><=;|&(:
$ COMP_WORDBREAKS=${COMP_WORDBREAKS//:}
$ echo $COMP_WORDBREAKS
"'><=;|&(

I am using arch linux with bash 4.4.12

@PaNaVTEC Strange that you're having the issue on arch with bleeding edge bash. I'm also on arch linux most of the time and I do not have to add that...
Anyway, since this seems to be a common and recurring theme here, I'll probably just locally update COMP_WORDBREAKS during script execution and then revert it back before exiting so that this is fixed across the board.
I'll try to get to this when I get time. PRs also accepted.

Thanks for the workaround!

This should work now without having to adjust your system's globally set COMP_WORDBREAKS variable.

I just installed yarn completion using curl -L https://raw.githubusercontent.com/dsifford/yarn-completion/master/yarn-completion.bash > brew --prefix/etc/bash_completion.d/yarn and unfortunately experienced this bug. Using standard macos bash terminal and the downloaded file is v0.5.1.
With build and build:watch scripts in package.json, here's what happens:
type yarn bui + hit tab
↓
yarn build + beep
type :w + hit tab
↓
yarn build:build:watch
What could be causing such a behaviour?

@kachkaev very strange that you still needed to set COMP_WORDBREAKS globally. Setting it locally should overwrite the local value in the execution context, so no idea why that didn't happen in your case.
Thanks for the update.

Can it be that it needs to be export COMP_WORDBREAKS=... instead of just COMP_WORDBREAKS=...?

That would definitely work, but I don't think that is something that should be done by a third party script since that change will leak across the entire OS and could potentially break something.

Hey @kachkaev I think I might know why you were having those strange issues.
What's the output when you enter bind -v | grep skip in your terminal?

@dsifford it's empty 🤔
macOS High Sierra 10.13.3 / bash

Hm...
Not 100% sure here, but it's clear that you're still using bash 3 (which, at the time of writing, is 9 years outdated).
In Bash 4, along with a host of other niceties, the readline library provides a variable called skip-completed-text which, when tabbing though completions in the middle of a word, skips the text you've already typed.
The reason why Mac OS is stuck at Bash 3 is because bash 4 changed to GPL v3 and mac can't ship that.
So, in short, to fix this issue for sure, and to make your life a whole lot simpler, I'd recommend just installing bash 4 using homebrew.

Hey @dsifford I'm using Archlinux as mentioned before in this issue, it ships with Bash 4.4 and I still have the same issue. Let me know if I can test anything for you
bash --version
GNU bash, version 4.4.19(1)-release (x86_64-unknown-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Output of bind -v:
$ bind -v | grep skip
set skip-completed-text off

Well, shoot. I have no idea then.
Are your dotfiles published online somewhere?

Sure, https://github.com/panavtec/dotfiles

Cool... I'll take a look a bit later.
This is kind of a hail mary, so no promises that I actually turn up with something.

 My bash --version on macOS was saying:
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17)
Copyright (C) 2007 Free Software Foundation, Inc.

After brew install bash and opening a new terminal, I got:
GNU bash, version 4.4.19(1)-release (x86_64-apple-darwin17.3.0)
Copyright (C) 2016 Free Software Foundation, Inc.

Although I still don't see any output in bind -v | grep skip, autocompletion after : works!

@kachkaev Glad to hear.
Make sure you follow the instructions that homebrew prints out after installing (related to changing your login shell) so that the changes persist through logins/logouts.
Otherwise, you'll be back with bash 3 next time you log in.

Not sure what I'm doing wrong, but this still isn't working for me. I installed it through Homebrew following the steps exactly. bash --version yields GNU bash, version 4.4.19(1)-release (x86_64-apple-darwin16.7.0).
I've run bind -v | grep skip and that didn't fix it either.
Example of what I mean:

yarn lint: (should have :ts, :sass, :html as valid options)
yarn lint:s TAB
yarn lint:s

@Nxt3 Did you make sure to set bash 4 as your login shell?
bash --version might print 4.4.19 if bash is available in your PATH, but if it's not your login shell, the completions won't work.

Nope 🙃 Sorry for wasting your time.

Haha not a waste of time at all @Nxt3!
I only knew to suggest that because I've been in your exact position in the past. 😊
Glad it's now working well for you!

@dsifford I think I spoke too soon. I've done a full reboot and it still isn't working. :|

Just the colon issue again or something else?
If just the colon issue, add COMP_WORDBREAKS=${COMP_WORDBREAKS//:} to your .bash_profile or .bashrc and give that a try

That's not working either. It's still just the colon issue.

what's the output of echo "$COMP_WORDBREAKS" in your terminal?

Should work. No idea why it isn't to be honest.
Do you have a minimal package.json that I can try?

https://gist.github.com/Nxt3/e4a6f43833242b5405180518158191c6

Hm, that works fine for me on my macbook.
Assuming you have the latest version of the completions installed, correct.
Also, I don't think this is written in the README yet which is my fault, but you should be using the latest version of bash-completion from homebrew, which is bash-completion@2 and not bash-completion.
Can you confirm those things?

I'll upgrade to bash-completion@2 and let you know.
It's weird, because I have another script called yarn start:dist and suggestions on that one work. However, it is the only yarn start:* that is present--whereas the lint one has multiple--so maybe it has something to do with that.

@dsifford I've updated to bash-completion@2 and it still isn't working.

Not sure what the issue could be then unfortunately. I'm running a MBP with high sierra and it works fine on my machine

Would the yarn version matter at all? Only thing I can think of.


Trouble including drogon source code locally

The instructions to include the drogon source code locally say to add this to CMakeLists.txt:
add_subdirectory(third_party/drogon)
target_link_libraries(${PROJECT_NAME} PRIVATE drogon)

I used vcpkg install drogon on Windows, and checked that it worked OK.  I then ran vcpkg remove drogon and vcpkg remove trantor so I would not have multiple copies of drogon and trantor.
I cloned drogon and the trantor submodule.  third_party/drogon looks correct.  I then ran:
cmake -DCMAKE_TOOLCHAIN_FILE=C:/Source/repos/vcpkg/scripts/buildsystems/vcpkg.cmake ..

I used -DCMAKE_TOOLCHAIN_FILE=... because vcpkg needs to find prerequisites which were installed earlier.
CMake fails to find the Drogon package:
r : CMake Error at C:/Source/repos/vcpkg/scripts/buildsystems/vcpkg.cmake:582 (_find_package):
At line:1 char:1
+ r 37
+ ~~~~
    + CategoryInfo          : NotSpecified: (CMake Error at ..._find_package)::String) [Invoke-History], RemoteException
    + FullyQualifiedErrorId : NativeCommandError,Microsoft.PowerShell.Commands.InvokeHistoryCommand

r :   Could not find a package configuration file provided by "Drogon" with any
At line:1 char:1

I am not very experienced with CMake.  I see some output in build/third_party/drogon, but not a complete build.
What did I do wrong?

Please use visio studio open the cmake project and compile it. (set the CMAKE_TOOLCHAIN_FILE in vs)

Please use visio studio open the cmake project and compile it. (set the CMAKE_TOOLCHAIN_FILE in vs)

I get this error in Visual Studio:
1> [CMake] -- Looking for C++ include string_view
1> [CMake] -- Looking for C++ include string_view - found
1> [CMake] CMake Error at C:/Source/repos/vcpkg/scripts/buildsystems/vcpkg.cmake:582 (_find_package):
1> [CMake]   Could not find a package configuration file provided by "Drogon" with any
1> [CMake]   of the following names:
1> [CMake]
1> [CMake]     DrogonConfig.cmake
1> [CMake]     drogon-config.cmake
1> [CMake]
1> [CMake]   Add the installation prefix of "Drogon" to CMAKE_PREFIX_PATH or set
1> [CMake]   "Drogon_DIR" to a directory containing one of the above files.  If "Drogon"
1> [CMake]   provides a separate development package or SDK, be sure it has been
1> [CMake]   installed.
1> [CMake] Call Stack (most recent call first):
1> [CMake]   CMakeLists.txt:26 (find_package)
1> [CMake] -- Configuring incomplete, errors occurred!
1> [CMake] See also "C:/Source/drogon/hello_world/out/build/x64-Debug (default)/CMakeFiles/CMakeOutput.log".

This is my directory structure:
.
├── CMakeLists.txt
├── build
├── config.json
├── controllers
├── filters
├── main.cc
├── models
│   └── model.json
├── plugins
├── third_party
│   └── drogon
│       ├── CMakeLists.txt
│       ├── CONTRIBUTING.md
│       ├── ChangeLog.md
│       ├── Dockerfile
│       ├── LICENSE
│       ├── README.md
│       ├── README.zh-CN.md
│       ├── README.zh-TW.md
│       ├── _config.yml
│       ├── build.sh
│       ├── cmake
│       ├── cmake_modules
│       ├── config.example.json
│       ├── drogon.jpg
│       ├── drogon_ctl
│       ├── examples
│       ├── format.sh
│       ├── lib
│       ├── orm_lib
│       ├── test.sh
│       ├── third_party
│       ├── trantor
│       └── unittest
└── views

Can you see my mistake?

please paste your CMakeLists.txt file of your project here.

please paste your CMakeLists.txt file of your project here.

My project is just the sample from drogon_ctl.txt create project with 2 additional lines.
cmake_minimum_required(VERSION 3.5)
project(hello_world CXX)

include(CheckIncludeFileCXX)

check_include_file_cxx(any HAS_ANY)
check_include_file_cxx(string_view HAS_STRING_VIEW)
if(HAS_ANY AND HAS_STRING_VIEW)
  set(CMAKE_CXX_STANDARD 17)
else()
  set(CMAKE_CXX_STANDARD 14)
endif()

set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

add_executable(${PROJECT_NAME} main.cc)

# ##############################################################################
# If you include the drogon source code locally in your project, use this method
# to add drogon
# add_subdirectory(drogon)
# target_link_libraries(${PROJECT_NAME} PRIVATE drogon)
# ##############################################################################

add_subdirectory(third_party/drogon)
target_link_libraries(${PROJECT_NAME} PRIVATE drogon)

find_package(Drogon CONFIG REQUIRED)
target_link_libraries(${PROJECT_NAME} PRIVATE Drogon::Drogon)

if(CMAKE_CXX_STANDARD LESS 17)
  # With C++14, use boost to support any and string_view
  message(STATUS "use c++14")
  find_package(Boost 1.61.0 REQUIRED)
  target_include_directories(${PROJECT_NAME} PRIVATE ${Boost_INCLUDE_DIRS})
else()
  message(STATUS "use c++17")
endif()

aux_source_directory(controllers CTL_SRC)
aux_source_directory(filters FILTER_SRC)
aux_source_directory(plugins PLUGIN_SRC)
aux_source_directory(models MODEL_SRC)

drogon_create_views(${PROJECT_NAME} ${CMAKE_CURRENT_SOURCE_DIR}/views
                    ${CMAKE_CURRENT_BINARY_DIR})
# use the following line to create views with namespaces.
# drogon_create_views(${PROJECT_NAME} ${CMAKE_CURRENT_SOURCE_DIR}/views
#                     ${CMAKE_CURRENT_BINARY_DIR} TRUE)

target_include_directories(${PROJECT_NAME}
                           PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}
                                   ${CMAKE_CURRENT_SOURCE_DIR}/models)
target_sources(${PROJECT_NAME}
               PRIVATE
               ${SRC_DIR}
               ${CTL_SRC}
               ${FILTER_SRC}
               ${PLUGIN_SRC}
               ${MODEL_SRC})
# ##############################################################################
# uncomment the following line for dynamically loading views
# set_property(TARGET ${PROJECT_NAME} PROPERTY ENABLE_EXPORTS ON)

find_package(Drogon CONFIG REQUIRED)
target_link_libraries(${PROJECT_NAME} PRIVATE Drogon::Drogon)

remove these two lines because you added drogon locally.

find_package(Drogon CONFIG REQUIRED)
target_link_libraries(${PROJECT_NAME} PRIVATE Drogon::Drogon)

remove these two lines because you added drogon locally.

Thank you.  That was it.  I am a beginner with CMake, so I did not know I needed to do that.


Download from eclipse doesn't work any more

The download of the language server from eclipse doesn't work any more, it probably got moved somewhere and the original link redirects. To have the installation work again, the -L flag has to be added to the respective curl calls. I did it here and here in my fork of your repository although I didn't try if the second one was necessary.

Didn't have the time to test this, do you mind opening a PR I can review later?

Thank you for detecting this and making the PR @yngwi

You're welcome. Thanks for making the repository :)


Test Coverage report is not available.

add jacoco plugin to the generate test coverage report.

Fixed in #10


R scan() function does not work

Hi,
Im trying to run this code on beaker notebook:
J <- 3
K <- 5
X <- scan(n=JK)
99.3 93.7 103.9 92.5 110.6
112.5 108.3 118.0 99.4 111.8
129.2 140.9 108.3 105.0 116.6
j <- rep(1:J, each=K)
j <- as.factor(j)
X.bar <- mean(X)
Xj.bar <- tapply(X, j, mean)
MSB <- sum((Xj.bar-X.bar)^2) * K / (J-1)
MSW <- sum((X-rep(Xj.bar,each=K))^2)/J/(K-1)
print(MSB/MSW)
print(qf(0.95, J-1, J(K-1)))
print(1-MSW/MSB)
And then I got this error:
Error
eval failed, request status: R parser: syntax error
Is there any issue with scan function???, this code works fine in R or R studio.
Thanks!

Scan() works if it's set to read from a file or the web.
But you are right, scan() fails if you try to read from stdin.
Note though I tried this code in RStudio and it fails also....
This might be hard to fix but we will try.
Thanks for the report.

https://github.com/ipython/ipython/issues/8787


⚠ Cioos National CKAN has degraded performance

In a57fd3d, Cioos National CKAN (https://catalogue.cioos.ca/) experienced degraded performance:

HTTP code: 200
Response time: 6073 ms

Resolved: Cioos National CKAN performance has improved in fd8669f after 5 minutes.


⚠ Adacta-fintech.com has degraded performance

In dc55ad3, Adacta-fintech.com (https://www.adacta-fintech.com) experienced degraded performance:

HTTP code: 200
Response time: 1083 ms

Resolved: Adacta-fintech.com performance has improved in b9d8341 after 9 minutes.


Run StackHPC OpenStack tests in aio CI

This change adds a new playbook, stackhpc-openstack-tests.yml, that uses
the new stackhpc-openstack-tests repository to perform testing of the
deployment that is not covered by Rally or Tempest. Currently this
includes Prometheus and OpenSearch.
The all-in-one CI test has been updated to run this new test suite, and
the results are included in the test results artifact.

yay


Ability to auto hide and reveal Touch Bar

It will be great to be able to auto hide and reveal virtual touch bar like dock!

Yes, that would be useful. We could add a setting to do this when it's docked to the top or bottom of the screen.

@issuehunt has funded $100.00 to this issue.

Submit pull request via IssueHunt to receive this reward.
Want to contribute? Chip in to this issue via IssueHunt.
Checkout the IssueHunt Issue Explorer to see more funded issues.
Need help from developers? Add your repository on IssueHunt to raise funds.


[swift] todolist / api tracking issue

Checked directly against Java.


Validating this certificate is not possible using dns-01. Possible validation methods are: http-01

I have a certificate requested through http-01, but want to convert it to dns-01 (wildcard). Dehydrated/letsencrypt gives me this error:
Validating this certificate is not possible using dns-01. Possible validation methods are: http-01

Other wildcard domains are working on my system... is there something I am missing?

Are you using the --force option on your renewal?
If you have already authorized a domain (and the authorization is still valid), then you are not allowed to change the validation method. (See https://github.com/win-acme/win-acme/issues/1399#issuecomment-587485998)
You could wait until the authorization expires (for Let's Encrypt, this will take 30 days), or you might try forcing the renewal, which I think will force a new authorization, which would allow your choice of validation type.

Are you using the --force option on your renewal?
If you have already authorized a domain (and the authorization is still valid), then you are not allowed to change the validation method. (See win-acme/win-acme#1399 (comment))
You could wait until the authorization expires (for Let's Encrypt, this will take 30 days), or you might try forcing the renewal, which I think will force a new authorization, which would allow your choice of validation type.

Yeah force is also giving same result. Maybe revoke the cert first?

I don't think revoking it is going to be a good idea--that would revoke the certificate, not the authorization...
@lukas2511 This might require a look?

Given that authorisations are attached to accounts, although it may not be an ideal solution, you could try deleting the existing account data, and creating a new account with ./dehydrated --register and then proceeding from there.

I added a new unused domain to my dehydrated setup with wildcard. In the certs directory I added a config file with this:
dehydrated/certs/example_nl/config
CHALLENGETYPE="dns-01"
HOOK="${BASEDIR}/hook_dns.sh"
KEY_ALGO=rsa

My domains.txt is this
example.nl *.example.nl > example_nl

But when running dehydrated I get this error:
dehydrated -c -g

Processing example.nl with alternative names: *.example.nl
 + Using certificate specific config file!
   + CHALLENGETYPE = dns-01
   + HOOK = /etc/dehydrated/hook_dns.sh
 + Signing domains...
 + Generating private key...
 + Generating signing request...
 + Requesting new certificate order from CA...
 + Received 2 authorizations URLs from the CA
 + Handling authorization for example.nl
 + Handling authorization for example.nl
 + 2 pending challenge(s)
 + Deploying challenge tokens...

Error opening Certificate /etc/dehydrated/certs/example_nl/rsa/cert.pem
140029208692624:error:02001002:system library:fopen:No such file or directory:bss_file.c:402:fopen('/etc/dehydrated/certs/example_nl/rsa/cert.pem','r')
140029208692624:error:20074002:BIO routines:FILE_CTRL:system lib:bss_file.c:404:
unable to load certificate

@it-can That looks like an error in your hook script, to me. Which should probably be handled as a separate issue.

@it-can That looks like an error in your hook script, to me. Which should probably be handled as a separate issue.

The cert.pem file is not there...

my dns script had a bug... maybe my dns problem is now also fixed

my dns problem is also fixed now....


docs: update compactType props on README.md

Description
Add null to compactType.
Because It was missing.

What type of PR is this? (check all applicable)

[ ] :pizza: Feature
[ ] :bug: Bug Fix
[x] :memo: Documentation Update
[ ] :art: Style
[ ] :technologist: Code Refactor
[ ] :fire: Performance Improvements
[ ] :white_check_mark: Test
[ ] 🤖 Build
[ ] :repeat: CI
[ ] :package: Chore (Release)
[ ] :fast_forward: Revert

Mobile & Desktop Screenshots/Recordings

Before
After

Added tests?

[ ] :+1: yes
[x] :no_good: no, because they aren’t needed
[ ] :raising_hand: no, because I need help

Added to documentation?

[x] :scroll: README.md
[ ] :notebook: examples
[ ] :no_good: no documentation needed

Fixed in #1973


Update TestScript

Coverage remained the same at 92.586% when pulling 9f030ce05255b02545ec1cc13e752473704af964 on johnweikangong:Documentation into 31d09d497bb908af411f7cde5858616e9337a81c on CS2103AUG2017-T16-B3:master.