Cannot apply parameter sharing on WeightDropParameter

Description
Parameter sharing of WeightDropParameter is broken. I apply weight sharing between AWDRNN and train.AWDRNN as followed:
model = nlp.model.train.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                               args.tied, args.dropout, args.weight_dropout,
                               args.dropout_h, args.dropout_i, args.dropout_e)
model_eval = nlp.model.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                              args.tied, args.dropout, args.weight_dropout,
                              args.dropout_h, args.dropout_i, args.dropout_e,
                              params=model.collect_params())

model.initialize(mx.init.Xavier(), ctx=context)

model.hybridize(static_alloc=True)

print(model)

def check_initialized(net):
    params = net.collect_params()
    for param in params:
        try:
            params[param].list_ctx()
        except RuntimeError:
            return False
    return True

print(check_initialized(model))
print(check_initialized(model_eval))

Log Message
After I ran the above code, I got the following message:
True
False

It appeared that model_eval is not properly initialized. After an inspection, we found it is the WeightDropParameter that not initialized.
(Pdb) model_eval.collect_params()["awdrnn0_hybridsequential0_embedding0_weight"].data()
*** RuntimeError: Parameter 'awdrnn0_hybridsequential0_embedding0_weight' has not been initialized. Note that you should initialize parameters and create Trainer with Block.collect_params() instead of Block.params because the later does not include Parameters of nested child Blocks
(Pdb) model_eval.collect_params()["awdrnn0_hybridsequential0_embedding0_weight"]
WeightDropParameter awdrnn0_hybridsequential0_embedding0_weight (shape=(33278, 400), dtype=float32, rate=0.1, mode=training)

Environment
My environment specs:
curl --retry 10 -s https://raw.githubusercontent.com/dmlc/gluon-nlp/master/tools/diagnose.py | python

---------Python Info----------
Version      : 3.6.6
Compiler     : GCC 7.2.0
Build        : ('default', 'Jun 28 2018 17:14:51')
Arch         : ('64bit', '')
------------Pip Info-----------
Version      : 19.2.3
Directory    : /home/ubuntu/anaconda3/lib/python3.6/site-packages/pip
----------MXNet Info-----------
Version      : 1.6.0
Directory    : /home/ubuntu/anaconda3/lib/python3.6/site-packages/mxnet
Num GPUs     : 1
Hashtag not found. Not installed from pre-built package.
----------System Info----------
Platform     : Linux-4.15.0-1056-aws-x86_64-with-debian-buster-sid
system       : Linux
node         : ip-172-31-23-26
release      : 4.15.0-1056-aws
version      : #58-Ubuntu SMP Tue Nov 26 15:14:34 UTC 2019
----------Hardware Info----------
machine      : x86_64
processor    : x86_64
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  2
Core(s) per socket:  4
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               79
Model name:          Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
Stepping:            1
CPU MHz:             2704.026
CPU max MHz:         3000.0000
CPU min MHz:         1200.0000
BogoMIPS:            4600.12
Hypervisor vendor:   Xen
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            46080K
NUMA node0 CPU(s):   0-7
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single pti fsgsbase bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx xsaveopt
----------Network Test----------
Setting timeout: 10

One workaround to the above problem is to initialize the model parameters before applying weight sharing:
model = nlp.model.train.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                               args.tied, args.dropout, args.weight_dropout,
                               args.dropout_h, args.dropout_i, args.dropout_e)
model.initialize(mx.init.Xavier(), ctx=context)
model_eval = nlp.model.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                              args.tied, args.dropout, args.weight_dropout,
                              args.dropout_h, args.dropout_i, args.dropout_e,
                              params=model.collect_params())

I suggest we drop the WeightDropParameter. Instead users can manually call Dropout in forward.
Understanding WeightDropParameter requires intricate familiarity with the way Gluon handles Parameters and Blocks, and most users will have a hard time to understand it. Further, as pointed out by @liuzh91, the implementation is currently broken.
What do others think?

The above workaround does not work if argument args.tied is set True.

The above workaround does not work if we have args.tied turned on. In this case, after I ran the code print(check_initialized(model_eval)), I got the following error message:
*** RuntimeError: Parameter 'awdrnn0_hybridsequential0_embedding0_bias' has not been initialized

It is exactly the output layer of the model_eval, it contains the following parameters:
model_eval.collect_params()
awdrnn0_ (
  WeightDropParameter awdrnn0_hybridsequential0_embedding0_weight (shape=(33278, 400), dtype=float32, rate=0.1, mode=training)
  Parameter awdrnn0_hybridsequential1_lstm0_l0_i2h_weight (shape=(4600, 400), dtype=float32)
  WeightDropParameter awdrnn0_hybridsequential1_lstm0_l0_h2h_weight (shape=(4600, 1150), dtype=float32, rate=0.5, mode=training)
  Parameter awdrnn0_hybridsequential1_lstm0_l0_i2h_bias (shape=(4600,), dtype=float32)
  Parameter awdrnn0_hybridsequential1_lstm0_l0_h2h_bias (shape=(4600,), dtype=float32)
  Parameter awdrnn0_hybridsequential1_lstm1_l0_i2h_weight (shape=(4600, 1150), dtype=float32)
  WeightDropParameter awdrnn0_hybridsequential1_lstm1_l0_h2h_weight (shape=(4600, 1150), dtype=float32, rate=0.5, mode=training)
  Parameter awdrnn0_hybridsequential1_lstm1_l0_i2h_bias (shape=(4600,), dtype=float32)
  Parameter awdrnn0_hybridsequential1_lstm1_l0_h2h_bias (shape=(4600,), dtype=float32)
  Parameter awdrnn0_hybridsequential1_lstm2_l0_i2h_weight (shape=(1600, 1150), dtype=float32)
  WeightDropParameter awdrnn0_hybridsequential1_lstm2_l0_h2h_weight (shape=(1600, 400), dtype=float32, rate=0.5, mode=training)
  Parameter awdrnn0_hybridsequential1_lstm2_l0_i2h_bias (shape=(1600,), dtype=float32)
  Parameter awdrnn0_hybridsequential1_lstm2_l0_h2h_bias (shape=(1600,), dtype=float32)
  Parameter awdrnn0_hybridsequential0_embedding0_bias (shape=(33278,), dtype=float32)
)

WeightDropParameter awdrnn0_hybridsequential0_embedding0_weight and  Parameter awdrnn0_hybridsequential0_embedding0_bias have their weights tied up.  WeightDropParameter awdrnn0_hybridsequential0_embedding0_weight is properly initialized whereas Parameter awdrnn0_hybridsequential0_embedding0_bias is not initialized.

If I applied weight dropout during the forward pass:
embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight']
embedding_param.set_data(mx.nd.Dropout(embedding_param.data(), 0.3))

I got the following error msg:
*** mxnet.base.MXNetError: [10:20:43] src/imperative/imperative.cc:203: Check fa
iled: AGInfo::IsNone(*output): Assigning to NDArrays that are already in a compu
tational graph will cause undefined behavior when evaluating gradients. Please c
all backward first to clear the graph or do this out side of a record section. A
lso note that you cannot use inplace operations like +=, *=, relu(x, out=x), y[i
dx]=x, etc inside a record section.

Note that my code is wrapped in the autograd.record().  The error suggests I apply the weight dropout outside the record section, but it is impractical in this case. Is there any workaround for this issue?

Yes, this will not work indeed.
You need to do something like
embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
embedding_param = mx.nd.Dropout(embedding_param, 0.3)

Yes, this will not work indeed.
You need to do something like
def forward(self, ...):
    ....
    embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    embedding_param = mx.nd.Dropout(embedding_param, 0.3)
    ....

It should uses slice assign embedding_param[:] instead of embedding_param.  Otherwise, the weight of network does not change at all. Concretely, I implement the following code:
    embedding_param = self.collect_params(['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    embedding_param[:] = mx.nd.Dropout(embedding_param, 0.3)

But if I do it in this way, I still get the same MXNetError.

I suggest we drop the WeightDropParameter. Instead users can manually call Dropout in forward.
Understanding WeightDropParameter requires intricate familiarity with the way Gluon handles Parameters and Blocks, and most users will have a hard time to understand it. However, our goal is to have easy to understand code. Thus in it's current implementation, WeightDropParameter doesn't align with our goals.
Further, as pointed out by @liuzh91, the implementation is currently broken.
What do others think?

It seems we cannot manually assign weights during the record session. I found a bunch of similar  assignment errors from the internet, e.g., https://discuss.mxnet.io/t/autograd-record-error-about-assigning-to-ndarray/322.  https://github.com/apache/incubator-mxnet/issues/9989
Perhaps we still need to investigate the apply_weight_drop() function and fix the case of shared parameters problem.

Why do you want to assign the weight? You can access the non-weight-dropped weight and apply dropout to the sliced part that you are interested. It's not necessary nor possible to modify the actual parameter via assignment in the forward pass.

Why do you want to assign the weight? You can access the non-weight-dropped weight and apply dropout to the sliced part that you are interested. It's not necessary nor possible to modify the actual parameter via assignment in the forward pass.
The code I posted above, which doesn't do any in-place modification of the weight is correct. You just need to keep the variable around so that you use the same dropped out version at any location in the forward pass.

The key problem is that how to do gradient backpropogation if the weight is not updated? Keep the network parameter in a temporary variable is also not memory efficient.

I think we can add the weight drop flag to the Dense/RNN/Embedding layers. In that case, we can apply dropout inside hybrid_forward:
https://github.com/apache/incubator-mxnet/blob/2ad3ce408b08456bff0b92a82aa3c484adde29f2/python/mxnet/gluon/nn/basic_layers.py#L222-L228

I finally figured out there is a mxnet weight sharing bug. When weight sharing is used with tied weight, something unexpected will occur. For this piece of code:
model = nlp.model.train.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                               args.tied, args.dropout, args.weight_dropout,
                               args.dropout_h, args.dropout_i, args.dropout_e)
model_eval = nlp.model.AWDRNN(args.model, len(vocab), args.emsize, args.nhid, args.nlayers,
                              args.tied, args.dropout, args.weight_dropout,
                              args.dropout_h, args.dropout_i, args.dropout_e,
                              params=model.collect_params())

model.initialize(mx.init.Xavier(), ctx=context)

model.hybridize(static_alloc=True)

Even I comment out all the apply_weight_drop in the code, the initialization error still occurred if args.tied is turned on. I printed out some pdb info here:
(Pdb) model_eval.decoder[0]._params
awdrnn0_hybridsequential0_embedding0_ (
  Parameter awdrnn0_hybridsequential0_embedding0_weight (shape=(33278, 400), dtype=float32)
  Parameter awdrnn0_hybridsequential0_embedding0_bias (shape=(33278,), dtype=float32)
)
(Pdb) model_eval.decoder[0]._params.list_ctx()
*** RuntimeError: Parameter 'awdrnn0_hybridsequential0_embedding0_bias' has not been initialized
(Pdb) model_eval.decoder[0]._params._shared
awdrnn0_hybridsequential0_embedding0_ (
  Parameter awdrnn0_hybridsequential0_embedding0_weight (shape=(33278, 400), dtype=float32)
)
(Pdb) model_eval.decoder[0]._params._shared.list_ctx()
[cpu(0)]

The conclusion is: no matter whether we use weight drop or not, weight tied and weight sharing cannot be used simultaneously. Otherwise, there will be an initialization error.
For the current weight drop bug, I think my fix is more simple and elegant. I will submit my PR very soon.

I think we can add the weight drop flag to the Dense/RNN/Embedding layers. In that case, we can apply dropout inside hybrid_forward:
https://github.com/apache/incubator-mxnet/blob/2ad3ce408b08456bff0b92a82aa3c484adde29f2/python/mxnet/gluon/nn/basic_layers.py#L222-L228

I think it is similar to @leezu 's proposal.  My point is that using weight drop in forward/hybrid_forward will lead to some unexpected gradient computation error. So I  suggest we stick to the existing implementation of apply_weight_drop.

Let me try that. It should be fine (otherwise we will need to fix it in the MXNet side). The autograd error you’ve met is due to the fact that the AG engine in MXNet does not support assignment operations.

The key problem is that how to do gradient backpropogation if the weight is not updated? If some weights are dropped in the forward pass, they should never be updated during current iteration. I really doubt whether the gradient computation will still work correctly with your hack.

It's the same as Dropout. The gradient will be 0 for the dropped out parts. The proposal above is correct.

It's the same as Dropout. The gradient will be 0 for the dropped out parts. The proposal above is correct.

I will check if gradient computation works correctly or not.

Ok, thank you. For your convenience, let me give a more complete code regarding above example:
If you had before
def forward(self, x):
    ....
    embeddings = self.embedding(x)  # self.embedding is a gluon.nn.Embedding
    ....

then for weight dropout you need to
def forward(self, x):
    ....
    embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    dropped_embedding = mx.nd.Dropout(embedding_param, 0.3)
    word_embedding = mx.nd.Embedding(x, dropped_embedding)
    ....

Further, you can optimize the computation by only performing dropout on the part of the weight that you are interested in:
def forward(self, x):
    ....
    embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    word_embedding = mx.nd.Embedding(x, embedding_param)
    dropped_embedding = mx.nd.Dropout(word_embedding, 0.3)
    ....

@leezu  Thanks for the example. I have a few concerns about the your code.
def forward(self, x):
    ....
    embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    word_embedding = mx.nd.Embedding(x, embedding_param)
    dropped_embedding = mx.nd.Dropout(word_embedding, 0.3)
    ....

I believe this piece of code is to apply dropout on the embedding instead of embedding weights.
original embedding architecture looks like:
embedding = nn.HybridSequential()
with embedding.name_scope():
    embedding_block = nn.Embedding(self._vocab_size, self._embed_size,
                                   weight_initializer=init.Uniform(0.1))
    if self._drop_e:
        apply_weight_drop(embedding_block, 'weight', self._drop_e, axes=(1,))
    embedding.add(embedding_block)
    if self._drop_i:
        embedding.add(nn.Dropout(self._drop_i, axes=(0,)))
return embedding

So I think your implementation should be like
def forward(self, x):
    ....
    embedding_param = self.collect_params()['awdrnn0_hybridsequential0_embedding0_weight'].data(CTX)
    if self._drop_e:
        embedding_param = mx.nd.Dropout(embedding_param, self._drop_e)
    word_embedding = mx.nd.Embedding(x, embedding_param)
    if self._drop_i:
        word_embedding = mx.nd.Dropout(word_embedding, self._drop_i)
    ....

Each time if there is a dropout operation, do you have to reimplement the block structure in the forward function? It looks really impractical.

Yes, above example for applying dropout sparsely was not correct. You would need the np.unique with return_inverse=True to make sure the same dropout mask is applied to repeated words.
But in fact, this kind of optimization is not crucial (it is currently not done either).
You're right about rnn cells. One way to avoid re-implementing the rnn cell is to add weight_drop support to the cells upstream in mxnet as @sxjscience suggested (https://github.com/dmlc/gluon-nlp/issues/1060#issuecomment-569169629).
So for now it may be simplest to keep apply_weight_drop around for the RNN use-case? When redesigning parameter handling in MXNet 2 we can revisit the design of apply_weight_drop and simplify the weight drop API / implementation.


Scheduled bans executed too early

It appears that there is a minor issue with the SQL query responsible for retrieving expired scheduled bans in the DWCScheduledBanRepository.ts file. The query seems to be in the wrong order, leading to unintended behaviour. To maintain the contributor's dignity, we won't mention their name or highlight this as a bug. In fact, let's all pretend this never happened.
https://github.com/HeadTriXz/Barry/blob/fe5f499ab665d84431fd3de34952a7ea46757da3/apps/barry/src/modules/moderation/database/DWCScheduledBanRepository.ts#L105

Note
Luckily, only one person has been affected by this issue.

Scheduled bans executed too early
That's an eloquent way to word instant ban

Scheduled bans executed too early.

That's an eloquent way to word instant ban

Luckily, only one person was affected by this issue.


chore: bdd-spring-1573650536 to 0.0.1

chore: Promote bdd-spring-1573650536 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Lumen support

It would be very useful update.

Hi, @djereg.
Do you think that you can collaborate with that?
PR are welcome.

Thanks.


pyparsing porting guide

It would be great to see a tutorial on how to port a pyparsing based project to Lark.

Hi, Is this issue still open. Would like to work on this issue :)

@Mou887 Absolutely! We'll welcome a porting guide.

can i work on this issue?

@osman-haider  Yeah, that would be great!
The docs support either markdown or RST.
I suggest choosing a minimal example that shows all the important tricks, and building the explanation around it.
Let me know if you have any questions. We can also set a chat on gitter.


Can Not Find Docker

System: Raspberry Pi 3
OS: LibreElec 9.2.6
Docker Plugin Installed: <IP_ADDRESS>
Problem: Install script says "Can Not Find Docker" even though Docker is installed and ps aux shows dockerd as running

I think this might be a path issue.  Did your ssh session where you are running install.sh start prior to installing the Docker add-on in Kodi?  If so, please try opening a new ssh session (close the existing one completely) and rerun install.sh.
What I think happened here:
When you open a new terminal session LibreElec will execute /etc/profile.d/99-kodi.conf.  This loops through all the installed Kodi addons and puts them on the users path so they can be called from the command line.  This would not get re-executed in your current session if you were to go into Kodi and install a new addon.  I can make a tweak to the installer script to re-execute this step so this doesn't happen in the future.

solved. thank you.


Why fetch data for all supported versions of ubuntu?

It says to run the following command to fetch data for all versions of Ubuntu.
goval-dictionary fetch-ubuntu 12 14 16

Why is that? Why not just fetch the data for the specific edition of Ubuntu one is running?
TIA

That's just an example of how to fetch multiple with one command. I only fetch 16 as that is the only version I run.

I think that should be made explicit in the doc.


Add 0.20 patch for Android

The MainActivity.java have a small change in RN 0.20 (see https://github.com/facebook/react-native/commit/abdca047b0d3a1f9627a485c15fad0b99c5c241e).
Let android/getPrefix.js support multi versions.

Wow! Thanks a lot! cc @grabbou

I like that!


Cannot enter lemurs interface

At the beginning, it worked well. But ,after about one weeks using, I figure that I cannot enter the lemurs interface but enter the tty1 directly (I set lemurs running in tty1).
After read the log, I figure that it seems that lemurs even don't run at all because there are no new logs in /var/log/lemurs.log. But it says that lemurs is active when I check the lemurs.service status
So, there are no logs I can provide.
What's wrong with it?

I install it from AUR
I modify the TTY in configure file, and it doesn't work in tty2.
Yes, after modifying configuration, I also disable display-manager.service and modify tty in lemurs.service, then I enable it.

For more information, before today, it just happened sometimes, not every time. I worked normally after rebooting.  But today, after I modifying the 10-monitor.conf in /etc/X11/xorg.conf.d/ and rebooting, it cannot work normally anymore. I don't know if it related to the bug. Because after I delete the configuration modified before, it still cannot work.

It become normal after I removed the version installed via AUR and  compiled by myself. Maybe the version in AUR have something wrong.But anyway, it works well now. THX


eslint security configuration is not passed to language server properly

From looking at my language server logs, I see that the eslint security configuration is getting passed as securities = { "warning", "error" }, when it should be passed as securities = { "1" = "warning", "2" = "error" }. I'm not sure where things are going wrong, but it seems like something (neovim, nvim-lspconfig, this plugin?) can't distinguish between a map and an array here.

Can you show how you have it configured? I've tried it on my end and not seeing any issue, able to get both errors and warnings. The securities is not ordered in the config but the index is correct. For reference I have tried with the following:
local dls = require 'diagnosticls-configs'
dls.setup {
  javascript = {
    linter = require 'diagnosticls-configs.linters.eslint'
  },
  typescript = {
    linter = require 'diagnosticls-configs.linters.eslint'
  }
}

Sure, here's a link to my vimrc: https://github.com/euclio/vimrc/blob/master/vimrc#L491-L505
I get both errors and warnings, but the severity is reported as "1" for all of them according to my LSP logs.

Fixed! Thank you.


Add bpf metics plugin

This PR adds support for collecting:

block IO latencies
runqueue latencies and
syscall latencies
as histogram metrics on Linux systems with kernel versions 4.3+ (in particular Ubuntu 16.04).

This branch is based on PR #108.

Removed chmod +s from Makefile, to avoid installing setuid executables on all hosts running nad.
Provide copy-paste-able installation instructions for Ubuntu 16

[x] Tested omnibus install on fresh Vagrant boxes: u14, u16, c6, c7.
[x] Tested bpf install instructions on u16.

Got the OK from @maier to merge this.


UPPERCASE_NAME replacement in stubs

I'm using custom stubs for module generation and in there I need UPPERCASE_NAME replacement. In simple worlds, I need to replace uppercase version of module name in stubs file. How I can do it. I've searched files but It seems there is no replacement function for it in src/Generators/ModuleGenerator.php file.
Please help me!

I've built a package for generating modules from a template this may be better for your needs https://github.com/dcblogdev/laravel-module-generator


Update Post “pre-purchase-termite-inspection-gold-coast”

Automatically generated by Netlify CMS

👷 Deploy Preview for chic-shortbread-66a66b processing.

Name
Link

🔨 Latest commit
25c484680b3a44b3a4793fcec9c14c1ccacea2cc

🔍 Latest deploy log
https://app.netlify.com/sites/chic-shortbread-66a66b/deploys/6540a8739f429e0008195da7


v1.3 scan: Master: Med 2

CampDv1.3APIupdate.xlsx
DNS Rebinding allows a malicious host to spoof or redirect a request to an alternate IP address,
potentially allowing an attacker to bypass security authentication or authorization.  DNS resolution
on its own does not properly constitute a valid authentication mechanism.  Servers should validate that
the Host header of the request matches the expected hostname of the server.  In cases where the hostname
is missing or does not match the expected value, the server should return a 400.  The X-Forwarded-Host
header is sometimes used instead of the Host header in cases where the request is being forwarded.  In
these cases, the X-Forwarded-Host header should also be validated if it is being used to determine the
Host of the original request.
DNS Rebinding on 'https://api.epa.gov/easey/staging/master-data-mgmt/account-type-codes?exclude=CASURR'

I did able to recreate the scan issue.
Add HOST in header key and gives the different value instead of ‘api.epa.gov’ then it’s returning the 404 error.
it’s not hitting in the application routes and not shows any logs.
Nginx throw 404 error instead of 400 error.

False Positive. Approved reclassification on 3/4/2024


Fix the bug that the atlas loading does not support relative paths

link #1478

Codecov Report
Patch and project coverage have no change.

Comparison is base (71dd283) 51.82% compared to head (74b49a6) 51.82%.

:exclamation: Current head 74b49a6 differs from pull request most recent head 36917cf. Consider uploading reports for the commit 36917cf to get more accurate results

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1535   +/-   ##
=======================================
  Coverage   51.82%   51.82%
=======================================
  Files         365      365
  Lines       18900    18900
  Branches     2720     2720
=======================================
  Hits         9795     9795
  Misses       8147     8147
  Partials      958      958

Impacted Files
Coverage Δ

packages/loader/src/SpriteAtlasLoader.ts
14.00% <ø> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


[RAC] Radio - pass ref to input element

Provide a general summary of the issue here
Looks like ref in radio is passed to label, not to the input element
🤔 Expected Behavior?
Ref should be forwarded to input element
😯 Current Behavior
Ref forwarded to label
💁 Possible Solution
Change forward ref logic, or add inputRef prop if passing default ref is not possible.
🔦 Context
I think users should have access to underlaying form element via ref. It is needed i.e. for react-hook-form
🖥 Steps to Reproduce
CodeSandbox
Version
"react-aria-components": "^1.1.1"
What browsers are you seeing the problem on?
Chrome
If other, please specify.
No response
What operating system are you using?
MacOS
🧢 Your Company/Team
No response
🕷 Tracking Issue
No response

We always forward the ref to the outer-most DOM element in every component. However, we recently added inputRef support to Checkbox/Switch in #5967, and probably should add the same thing to Radio as well.

@devongovett good, thanks. I think there is always should be ref access to form elements, just in cases like with react-hook-form.

I think this issue can be closed, now that https://github.com/adobe/react-spectrum/pull/6067 is merged?
Sorry I had a typo in the PR that didn't link to this issue properly before it got merged 😅 (Closes #6054, #6123 -> should have been -> Closes #6054, closes #6123)

@sookmax Yep, I think we can close this issue. Thanks for pointing it out!


Number of rows for current month?

With the latest build, I can see 6 rows per month, which is unlike normal calendar would do. I noticed that it's marked as closed (#168) but is still presenting.
Any ideas? or where should I look into to modify it myself?
Nice work btw!

Working on it now.
Thanks.

It looks like the bug is in -numberOfRowsInMonth: in FSCalendar+DateTools.m.
Line 72 has
if (self.showsPlaceholders) return 6;
If you comment out that line, the calendar renders correctly even when showing placeholders.


Connection refused from docker to docker on port 27022

Running mongo container on latest windows 10 update.
Docker (WSL 2) with ubuntu.
I have asp dot net core api CRUD applications that connect to mongo db and running as Docker.
Once i am running container  with 27017 port, everething is ok
docker run --name mongo -d -p **27017**:27017 mongo --bind_ip_all
but if i change to any other port 27019, 27022 etc
docker run --name mongo -d -p **27019**:27017 mongo --bind_ip_all
i get Connection refused <IP_ADDRESS>:27022
Why? :-(

If you're directly connecting to the container then the port forwarded using the -p option won't affect that connection, it only forwards that port through the host's firewall to allow for external connectivity to the container. Example https://github.com/docker-library/postgres/issues/650#issuecomment-562253607
You could also try asking over at the Docker Community Forums, Docker Community Slack, or Stack Overflow. Since these repos aren't really a user-help forum

Thank you!


custom instruction creations do not sync

Hi, first of all, thanks a lot for your brilliant plugin! :))
One little issue: custom instruction creations do not sync into my vault. There is just a heading called "custom" without any content underneath.

Thank you from the bottom of my heart! This bug, like others, should be resolved in the beta currently being rolled out! (CF : https://github.com/mysticcoders/voicenotes-sync/pull/42 ) 🦦

v0.0.20 fixes this


find_package and CPMFindPackage have different behaviors

Hello,
I'm using CPMFindPackage() to check for a dependency on CMocka and noticing some strange behavior with the find_package portion. Do you know what I might be doing wrong here?
CPMFindPackage(
  NAME CMOCKA
  GIT_REPOSITORY https://git.cryptomilk.org/projects/cmocka.git/
  VERSION 1.1.5
  GIT_TAG cmocka-1.1.5
  OPTIONS
    "WITH_EXAMPLES OFF"
)

message("CMOCKA_FOUND: ${CMOCKA_FOUND}")
message("CMOCKA_ADDED: ${CMOCKA_ADDED}")
message("CMOCKA_INCLUDES: ${CMOCKA_INCLUDE_DIR}")
message("CMOCKA_LIBRARIES: ${CMOCKA_LIBRARIES}")

The library is installed on my system with a CMake config file. I see this output from CPM, but the FOUND/ADDED variables are not set, and the other variables defined by the config file aren't used.
-- CPM: using local package<EMAIL_ADDRESS>CMOCKA_FOUND:
CMOCKA_ADDED:
CMOCKA_INCLUDES:
CMOCKA_LIBRARIES:

If I remove CPMFindPackage and add find_package:
find_package(CMOCKA)

This time, CMOCKA_FOUND will be defined, and the values defined by the config file are used.
CMOCKA_FOUND: 1
CMOCKA_ADDED:
CMOCKA_INCLUDES: /usr/local/include
CMOCKA_LIBRARIES: /usr/local/lib/libcmocka.dylib

I wonder if this is a scoping problem because find_package is called inside of a function.

Hey, thanks for the report! The problem is indeed that CPMFindPackage is a function and has it's own scope. Therefore, any variables exported to the parent scope by the FindXXX.cmake script get lost in the inner scope. This is expected behaviour.
Basically, CPMFindPackage guarantees that the dependency has been added after the call and we can use any targets defined inside. In modern CMake, the targets themselves should contain all libraries and include directories, so that no variables need to be exported.
If the dependency does depend on variables, I'm not really sure what we can do about this as usually the generated CMake module scripts (used by find_package) differ strongly from the source scripts (CMakeLists.txt) and there would be no good way to get reliable build behaviour.

Thanks for the explanation. Those are nice notes for the documentation too, I have a better understanding of the requirements for using CPMFindPackage
Perhaps this can be a suggested workaround in the documentation for cases where the dependency's config.cmake file doesn't export targets.
find_package(cmocka QUIET)

if(NOT cmocka_FOUND)
  CPMAddPackage(
    NAME cmocka
    GIT_REPOSITORY https://git.cryptomilk.org/projects/cmocka.git/
    VERSION 1.1.5
    GIT_TAG cmocka-1.1.5
    OPTIONS
      "WITH_EXAMPLES OFF"
  )

  # Maintain build compatibility between find_package and CMakeLists.txt variants
  set(CMOCKA_LIBRARIES cmocka)
endif()

If the dependency does depend on variables, I'm not really sure what we can do about this as usually the generated CMake module scripts (used by find_package) differ strongly from the source scripts (CMakeLists.txt) and there would be no good way to get reliable build behaviour.

Scoping aside, I think that's the user's responsibility. I work around this for the current dependency by setting a single variable when CPMAddPackage is used instead of find_package.
# Maintain build compatibility between find_package and CMakeLists.txt variants
set(CMOCKA_LIBRARIES cmocka)

Thanks for the feedback, I agree that this should be mentioned in the documentation. Also your user-side workaround makes sense, I'll see that we add this to the docs soon.

@TheLartians You could export those variables by inspecting the directory property https://cmake.org/cmake/help/latest/prop_dir/VARIABLES.html before and after the find_package call and promote any new variables to PARENT_SCOPE

I'm having a similar issue with a project "A" that gets Catch2 via CPM but also includes project "B" that gets Catch2 via Fetchcontent. When I configure the project, the FetchContent call in project B doesn't see that CPM has already got Catch2 so tries to get it again and fails with the following error:
The binary directory
[cmake]     /build/_deps/catch2-build
[cmake]   is already used to build a source directory.  It cannot be used to build source directory
[cmake]     /build/_deps/catch2-src
[cmake]   Specify a unique binary directory name.

The relevant section of the cmake file for project A is:
CPMAddPackage(
  NAME catch2
  GIT_REPOSITORY https://github.com/catchorg/Catch2
  VERSION 2.10.2
)

if(${catch2_ADDED})
  # Path to module "Catch" containing catch_discover_tests()
  list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/contrib)
  set( CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} PARENT_SCOPE)
endif()

And project B:
include(FetchContent)
FetchContent_Declare(catch2
GIT_REPOSITORY "https://github.com/catchorg/Catch2"
GIT_TAG 87b745da6625b7469e61de5910978a097caf8b74 # v2.10.2
)

FetchContent_GetProperties(catch2)

if(NOT catch2_POPULATED)
  FetchContent_Populate(catch2)
  add_subdirectory(${catch2_SOURCE_DIR} ${catch2_BINARY_DIR})
  # Path to module "Catch" containing catch_discover_tests()
  list(APPEND CMAKE_MODULE_PATH ${catch2_SOURCE_DIR}/contrib)
  set( CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH} PARENT_SCOPE)
endif()

I've tried setting CATCH2_LIBRARIES as per the above example and also tried setting catch2_POPULATED but this just gets set back to false by the call to FetchContent_GetProperties(catch2)

Hey @apGribble thanks for the comment, that is an interesting use-case. I think the issue could be due CPM.cmake bypassing FetchContent for performance reasons. Could you try disabling the cache by changing the first CPMAddPackage call to the following?
  CPMAddPackage(
    NAME catch2
    GIT_REPOSITORY https://github.com/catchorg/Catch2
    VERSION 2.10.2
++  NO_CACHE TRUE
  )

Yeah that removes the error.
Is there actually much overhead with calling fetchcontent when the SOURCE_DIR is in the local cache.

Yeah, each invocation of FetchContent currently comes with a significant (100 - 1000 ms) performance penalty, even if all sources are already cached. The CMake maintainers are aware of this and already working on a fix though.


Add CadQuery Action

There are usage examples in the following repos:

https://github.com/Yeicor/cadquery-action
https://github.com/Yeicor/3d-usb-pin-connector
https://github.com/Yeicor/3d-dishwasher-detergent-holder

Thanks @Yeicor !


Question: input data-role="search"

👋🏽 !!
Question
I'm using intput data-role="search" and i need when user clicks on "X" button clear the results
thanks

Clear input field

________________________________
From: jcpc91
Sent: Monday, July 16, 2018 4:15 AM
To: olton/Metro-UI-CSS
Cc: Subscribed
Subject: [olton/Metro-UI-CSS] Question: input data-role="search" (#1139)
👋🏽 !!
Question
I'm using intput data-role="search" and i need when user clicks on "X" button clear the results
thanks
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHubhttps://github.com/olton/Metro-UI-CSS/issues/1139, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AfHt67NnwsLcnVLYWssTihblPA0aSmQ2ks5uHBN-gaJpZM4VQix8.

I don't quite understand your question. What does it mean to clear the search result?
You can use custom buttons for input to create your one specific action


Inference code

Can you share the code which can be used to check the trained network inference on CityScape test data sets?

The inference code on Cityscapes test datasets is same to the eval script. Thus, you can use the eval script on the test dataset directly.

So should I put the dataset in a special dir, or just anywhere?


Allow aeson 1.5

Builds fine and all tests pass here.

Thank you. Do you mind doing DCO with git commit --amend --signoff & git push --force prior to merging?

Sure, updated :)

Cheers :) I've updated the version bound on hackage as well.


[Feat] Added AI copilot

A auto completion copilot which uses Llama3.1 by default which implements inline suggestions.

Add docker compose preset for ollama.

Development will be paused due to effectiveness issues.


Training Method for Stopping Product Cut-off

Hi @xuebinqin
Great job indeed.
Problem Statement: Input and output files are attached for your consideration. However, I have identified the problem area of the output. Please find attached. My question is how to train this sort of image so that it (product cut-off) does not happen next time. What would be the best way to do it? I look forward to hearing from you soon.
Thank you

Hi @xuebinqin
I look forward to hearing from you.
Thank you

You should train the model yourself with a custom dataset to get better results


upd(obs-backgroundremoval): 0.5.19 -> 1.0.0

CI fails for 2 reasons:

Ubuntu 22.04 and debian stable are incompatible
cmake-deb fails to uninstall


base64 to jpeg

Is there a way to decode the base64 cropper image into a jpeg file?

This solution worked for me: https://stackoverflow.com/questions/4998908/convert-data-uri-to-file-then-append-to-formdata


Labels are cut off sometimes

Describe the bug
I've a webmap with labels enabled and at times it's cut off
https://lgtest.mapsdevext.arcgis.com/apps/instant/reporter/index.html?appid=38b7b73ee28d45c4a1976335e0219c94
Expected Behavior
Label should be rendered fully.
Reproduction Steps & Sample
Click on the Layer with attachments > Click on feature 88

Other Relevant Info
No response

@skrishn
This is JS API's map behavior we don't have any control over it


feat(cli): capture unknown commands making use of the inhouse solution

fixes #1416

thank you @jamesgeorge007 !


chore: Add Jinja templating support

Hey @gvozdvmozgu I think we can park JJ01/Jinja support for now.
My 2 main concerns that we are only realising now:

Means we need to add the Jinja dependency into our code-base which we want to avoid for now
We don't fully know the compatibility between 'minijinja' and actual jinja which might lead to a risk further on.

Feel free to skip the JJ01 rule  for now, we will park this PR for now so feel free to skip. Great work nonetheless!


add info about community meetings to readme

When https://github.com/deislabs/smi-spec/pull/112 is merged, this issue can likely be resolved.


[Backport] HRTR: make pending task execution handling to go through all tasks on not finding worker slots (#8697)

backports #8697

Updated the title to help the missing backports script.

@jihoonson thanks, didn't know about that .. it would be great if that can be documented in CONTRIBUTING.md .

Sounds great. It would also be great if we have a script for backporting PRs. I could be able to work on these sometime.. probably.

oh, from your message I got the [wrong] idea that there was probably some script that was doing something with backported PRs and that is why titles had to follow a specific format .

Aha sorry for the confusion. You got the right idea. distribution/bin/find-missing-backports.py searches for all PRs which are in master but not in the given milestone branch. This works based on some pattern matching on the PR title.

Sounds great. It would also be great if we have a script for backporting PRs. I could be able to work on these sometime.. probably.

My comment was about a script backporting one PR in master into a branch. Gian shared his script before which is not completely configurable (for example, you have to fix the hard-corded user name in the script) and probably we can further generalize it.

got it, thanks


Travis

I started to look into Qiyun PR and was happy to see unit tests. However, it seems like we don't have Travis for this repo. We should definitively have it!

done!


consider using nose2 instead of nose

nose is unmaintained, and its earlier maintainer recommends moving on to use nose2 (see https://github.com/nose-devs/nose/pull/1004#issuecomment-260155457).

The homepage of nose lists several alternatives:

New projects should consider using Nose2, py.test, or just plain unittest/unittest2.

quoting from http://nose.readthedocs.io/en/latest/
where links are:

nose2: https://github.com/nose-devs/nose2
py.test: http://pytest.org/

pytest seems worth considering, it appears to be much more actively developed, and more widely used.

concerning differences between nose and nose2: https://nose2.readthedocs.io/en/latest/differences.html

In the tulip-control project, this issue also applies to

[ ] polytope
[ ] dd

I recommend pytest and will move forward with it. This might be useful for legacy code using nose: https://docs.pytest.org/en/latest/nose.html

nose is not compatible with Python 3.10 (due to imports from the module collections of classes that have moved to the module collections.abc). I recently (in a local repository) transitioned dd from nose to pytest, which informed me about its usage. I would be happy to transition tulip to pytest. I also transitioned to pytest the packages polytope (on branch py310) and omega (in a local repository).

Using pytest seems like a good path forward.

I will start the transition to using pytest.

The transition to pytest has been completed on branch use_pytest, not yet merged into the mainline branch. I also transitioned CI testing to GitHub Actions, as described in #247, because Travis CI is no more free.
So the branch use_pytest is I think nearly ready to be rebased and merged. I intend to wait a week or more, before merging this branch, in order to ensure that I have not forgotten anything.
Some remarks
There are various ways in which pytest is inequivalent to nose, including a number of items that pytest, as far as I know, does completely differently.
Where generated files are written
For example, with nose it is possible to run nosetools programmatically from a Python script above the directory tests/ and have the files generated by tests be within the directory tests/. I could not find any way of configuring pytest to do that, so I programmatically change directory to tests/ (depending on command-line options given to the script run_tests.py). I arranged this particular change as a separate commit, in case pytest changes in the future, and there is interest in the previous implementation.
Switching to the directory tests/ also ensures that only the installed copy of tulip is importable, i.e., it makes it impossible to import tulip from its source directory within the repository (well, assuming that path is not added to sys.path by Python code). Though this is possible to configure in pytest, it is an extra layer of safety. (In other packages, I have been cd tests/ before running tests). This way, even if something in pytest changes, importing will only be possible from the installed copy of tulip under site-packages.
Interference between test modules
Another observation (and there are several others), which actually does not apply to tulip, but was observed for dd, is that pytest runs tests in a way that separate Python test modules can interact.
For example, what happened in dd was that a number of test modules imported common test modules, and set class variables. With nose, this worked without interference between the two modules. With pytest, the imported modules are shared between the importing modules, which results in the class variables having the same value in the importing test modules.
In more detail, the test modules were:
tests
├── autoref_test.py  # imported modules `common`, `common_bdd`, and
    # set the class variables `common.Tests.DD` and `common_bdd.Tests.DD`
    # to the class `dd.autoref.BDD`
├── common.py
├── common_bdd.py
├── common_cudd.py
├── cudd_test.py  # imported modules `common`, `common_bdd`, `common_cudd`, and
    # set the class variables `common.Tests.DD`, `common_bdd.Tests.DD`, `common_cudd.Tests.DD`
    # to the class `dd.cudd.BDD`
├── cudd_zdd_test.py  # imported modules `common`, `common_cudd`, and
    # set the class variables `common.Tests.DD`, `common_cudd.Tests.DD`
    # to the class `dd.cudd_zdd.ZDD`

This was an issue in pytest, because that class variable determined which class was being tested. Thus, some of the test modules were actually testing the class of another dd module than what they were supposed to be testing. I noticed this by coincidence, when trying to fix something else while transitioning from nose to pytest.
To correct this issue, I turned the class attributes to data attributes (i.e., instance attributes), and instead of modifying the classes of tests imported from the modules of common tests (modules common, common_bdd, and common_cudd above), I now subclass them, and set the data attributes within the subclass method setup_method. This is the correct approach for ensuring separation between tests.
Even though in this particular case, my initial approach with class attributes was flawed, and using subclasses is a better approach, it is conceivable that there may be use cases where one would like to run tests in a separated way. The approach of pytest has the benefit that it will detect such interference, which in principle could be caused also by the package's code (in which case it would have been a bug in the package, and not just an issue with how the tests work). Nevertheless, there may be cases where the opposite behavior (i.e., separated tests) is needed.
(Also, the two plugin packages pytest-xdist and pytest-forked that are for running tests with parallelism and using subprocesses, exhibit the same behavior as pytest in terms of interference between different test modules.)

The transition to using pytest has been completed. Commit a198b5116887f241171540cae789e524b5f085ab is the main change, and together with followup changes is tested using GitHub Actions as of commit 7ececa4bcae51c85759e7678f706c160e0fc4fa5.


Remove out of date platforms (MacOS 10.14, CentOS8, Ubuntu16)

See also EPE 1125
Change Description

API Changes

[ ] API Changes

Documentation Additions

[ ] Documentation Additions

I didn't look for things you may have missed. All the changes look appropriate. I recommend grepping for other references locally.


Adding an "only" option

Hello,
However, for my project, I needed the ability to limit the resolution of node_modules to only modules matching a pattern, so I added this option. The implementation is very straightforward, using String.prototype.match.
Using it is very simple. It works with both strings and regexes:
{
  entry: 'samples/only/main.js',
  plugins: [
    nodeResolve({
      only: [ 'test' ]
    })
  ]
}

import test from 'test' // will be bundled
import test2 from 'test2' // /!\ will be bundled
import mytest from 'mytest' // will NOT be bundled

{
  entry: 'samples/only/main.js',
  plugins: [
    nodeResolve({
      only: [<EMAIL_ADDRESS>]
    })
  ]
}

import test from 'test' // will NOT bundled
import mytest from '@scoped/mytest' // will be bundled
import mytest2 from '@otherscope/mytest' // will NOT be bundled

Hey @arantes555 - this looks like a good PR. My question is, does jail not help you here?
Also this PR has some conflicts - so if you think we should really add this it would be great to get it rebased 😃

@keithamus : branch has been rebased, and some cleanup has been done :) I believe it's good to merge on my end!

@lukastaegert : I just pushed a minor edit with your suggestion !

3.3.0 😄


Support rdelta for datetimes

Part of  https://github.com/open-formulieren/open-forms/issues/2557

Codecov Report

Merging #13 (15ba710) into master (e0ccad0) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master      #13   +/-   ##
=======================================
  Coverage   94.16%   94.16%
=======================================
  Files           6        6
  Lines         257      257
  Branches       61       61
=======================================
  Hits          242      242
  Misses          8        8
  Partials        7        7

Impacted Files
Coverage Δ

json_logic/__init__.py
91.30% <100.00%> (ø)

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@SilviaAmAm feel free to bump the version and git tag that commit to release this to PyPI


Better  index to fetch job on procrastinate_jobs

Hello,
We noticed this query can be expensive:
SELECT id, task_name, lock, queueing_lock, args, scheduled_at, queue_name, attempts
    FROM procrastinate_fetch_job(ARRAY[$1])

This part of the function is expensive when there is several jobs:
        SELECT jobs.*
            FROM procrastinate_jobs AS jobs
            WHERE
                -- reject the job if its lock has earlier jobs
                NOT EXISTS (
                    SELECT 1
                        FROM procrastinate_jobs AS earlier_jobs
                        WHERE earlier_jobs.lock = jobs.lock
                            AND earlier_jobs.status IN ('todo', 'doing')
                            AND earlier_jobs.id < jobs.id)
                AND jobs.status = 'todo'
                AND (target_queue_names IS NULL OR jobs.queue_name = ANY( target_queue_names ))
                AND (jobs.scheduled_at IS NULL OR jobs.scheduled_at <= now())
            ORDER BY jobs.id ASC LIMIT 1
            FOR UPDATE OF jobs SKIP LOCKED

We can add this index to improve performance:
CREATE index concurrently procrastinate_jobs_queueing_lock_idx2  ON procrastinate_jobs  (lock,id) WHERE status = IN ('todo'::procrastinate_job_status,'doing'::procrastinate_job_status);

Note that it mostly happens when the finished jobs (status succeeded or failed) are not cleaned ([here] is how to delete them). As the table grows, the sequential scan in the inner loop takes more time. And because we are in an inner loop, it's exponential…

We just found another case when it's costly: where there is a lot of records in the table, and none are todo. The jobs scan kills the db (it still tries the index). So we also need this index:
create index CONCURRENTLY IF NOT EXISTS procrastinate_jobs_id_todo on procrastinate_jobs(id) WHERE status='todo';

it's been added in production… runtimes went from 200ms to 0.05ms

We factorized the 2 indexes into only one:
create index CONCURRENTLY procrastinate_jobs_queueing_lock_idx2
ON procrastinate_jobs (id, lock)
WHERE status = ANY (ARRAY['todo'::procrastinate_job_status, 'doing'::procrastinate_job_status]);


Python 3.11 support

Using the provided requirements.txt file, Pillow doesn't work with this error:
setup.py:45: RuntimeWarning: Pillow 9.0.1 does not support Python 3.11 and does not provide prebuilt Windows binaries. We do not recommend building from source on Windows.
        lambda: warnings.warn(
      [end of output]

error: legacy-install-failure

I'd like to add it does not work with Python 3.6 unlike what the Readme states for the exact same reason

Also unable to use this with Python 3.11. An update to the README to reflect this would be appreciated.

Hi all,
Sorry for the delay. Thanks for reporting the issue, this should be resolvable with some tweaking of the requirements.txt file.


support/db/dbtest: Remove postgresql-client-9.6 dependency for running tests

PR Checklist
PR Structure

[x] This PR has reasonably narrow scope (if not, break it down into smaller PRs).
[x] This PR avoids mixing refactoring changes with feature changes (split into two PRs
otherwise).
[x] This PR's title starts with name of package that is most changed in the PR, ex.
services/friendbot, or all or doc if the changes are broad or impact many
packages.

Thoroughness

[x] This PR adds tests for the most critical parts of the new functionality or fixes.
[ ] I've updated any docs (developer docs, .md
files, etc... affected by this change). Take a look in the docs folder for a given service,
like this one.

Release planning

[ ] I've updated the relevant CHANGELOG (here for Horizon) if
needed with deprecations, added features, breaking changes, and DB schema changes.
[ ] I've decided if this PR requires a new major/minor version according to
semver, or if it's mainly a patch change. The PR is targeted at the next
release branch if it's not a patch change.

What
The integration tests for horizon use os/exec to invoke psql, createdb, and dropdb to initialize postgres. Instead of invoking those executables we should use github.com/jmoiron/sqlx to initialize postgres.
Why
This allows us to remove the postgresql-client-9.6 dependency which was only necessary for running Horizon tests (it wasn't required for running the Horizon service). Removing this dependency makes it easier for new Horizon developers because now they are able to run Horizon tests without having postgres installed locally (postgres can be running in a docker container instead).
Known limitations
[N/A]

Is it possible to use the stdlib sql package instead of sqlx, does that work? What does SQLX give us in tests given we're just executing statements?

@leighmcculloch I think it should be possible to use the stdlib sql package. However, our tests are already coupled to sqlx as you can see below:
https://github.com/stellar/go/blob/master/support/db/dbtest/db.go#L43
https://github.com/stellar/go/blob/master/services/horizon/internal/test/main.go#L34
I kept using sqlx for the sake of consistency

Is it possible to use the stdlib sql package instead of sqlx, does that work? What does SQLX give us in tests given we're just executing statements?

@leighmcculloch I think it should be possible to use the stdlib sql package. However, our tests are already coupled to sqlx as you can see below:
https://github.com/stellar/go/blob/master/support/db/dbtest/db.go#L43
https://github.com/stellar/go/blob/master/services/horizon/internal/test/main.go#L34
I kept using sqlx for the sake of consistency


Multi providers integrations

Hello,
Is there a way to connect with two id providers. I mean i want to login on two different providers and my app is the service provider?

In Saml2 you have two strategy SP side:

Embed a wayf Page in the SP
Use a discoverysery service

Fortunately djangosaml2 have a wayf Page in It, Just put idp metadata and remove the idp definition used to connect to It. It works as It is

Hello @peppelinux thank you for the support, can you explain the above methods in more details please ?

Those methods are described in SAML2 OASIS official documentation.
Regarding djangosaml2 the integrated method in it is quite easy. Do not define any idp configuration in settings.
Add as many metadata resource as the authentication endpoints (IdP) are, djangosaml2 will show to the user a page from which a user can select the IdP to be authenticated to.
Share your example configuration if you can, I think that that's the best way to get an example to start working on

Im following the official docs and my settings configs looks something like this:
SAML_CONFIG = {
'allow_unknown_attributes': True,
'accepted_time_diff': 1800,
# full path to the xmlsec1 binary programm
'xmlsec_binary': '/usr/bin/xmlsec1',
# your entity id, usually your subdomain plus the url to the metadata view
"entityid": "http://mydomain.com/login_handler/",
# this block states what services we provide
'service': {
# we are just a lonely SP
'sp': {
'name': 'Federated Django sample SP',
"allow_unsolicited": True,
'endpoints': {
# url and binding to the assertion consumer service view
# do not change the binding or service name
'assertion_consumer_service': [
('https://example.com/login_handler/',
saml2.BINDING_HTTP_POST),
],
# url and binding to the single logout service view
# do not change the binding or service name
'single_logout_service': [
('https://example.com/saml2/ls/',
saml2.BINDING_HTTP_REDIRECT),
],
},
},
},
# where the remote metadata is stored
'metadata': {
'local': [os.path.join(BASE_DIR, 'metadata/idp_1.xml'), os.path.join(BASE_DIR, 'metadata/idp_2.xml')],
},
# set to 1 to output debugging information
'debug': 1,
"allow_unsolicited": True,
}

Doesn't It show to you a wayf Page where to select the idp?
You can even initialize It arbitrarly with
/saml2/login/?idp=$idp_entity_id&next=/saml2/echo_attributes
next is the Page where to be redirected to, from your ACS (sp's assertion consumer service)

Thanks for all the support @peppelinux im expirementing at the moment, will let you know if anything happens

is it ready to be closed and everythings works as you expect by your side?

@peppelinux yes thank you for the support


MP Loads Test Automation v1.0

Context
Create one script that complete the following actions for MP Loads  Create, Edit, Evaluate, Revert to Offical Record

Automated Test Complete


Full width row in F4E 2.0

Is it possible to achieve a full width row in F4E 2.0?  (Similar to swapping the .row and .container classes in F4E1)

Are you trying to a make a fluid width header or background?

Yes, a fluid width header.

Yes, it works the same way. If you wrap a table or a row around the container, you can do this. You might need to override the background of the container.

is it possible to do this using INKY?
<row>   <container>     <columns small="12" large="12">     full width     </columns>   </container> </row>
Does not seem to work

I found a solution by doing the following:
<row class="full-width"><container><row><columns></columns></row></container>	</row>

Brandon I also found at that you can do this using inky. Just wrap the container in a div with the class of header and boom!


feat: remove Roadhog heal buff from using Take a Breather - [merged]

Merges hog-vape -> staging

fixes #2

mentioned in commit 2a8787b4399ed9c87851cb9cff9bea87c1829a8c


[Little aestethic issue] Setting icon (gear) has no outline

the gear icon of setting has no outline so when an user has a white profile background the gear will vanish, so hard to see and find. i suggest to add a VERY thin black (or gray) outline to distinguish the icon in these cases. also the icon next to it has the same problem.

(to avoid github bot closing it)


Typo in "Licenses and subscriptions for business users"

It looks like you have a typo in the Licenses section
"Each user of the Power BI service has either a free license, a Pro license, of a Premium per-user* license." I presume this is meant to say or a Premium per-user* license rather then of... - but if not, then further clarification of the sentence's meaning would be helpful.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 696d95c8-9c63-06ad-3f7a-7d7e83519ffd
Version Independent ID: e92490bd-7b7f-2fdc-823a-46d81d39dc55
Content: Types of licenses for Power BI business users - Power BI
Content Source: powerbi-docs/consumer/end-user-license.md
Service: powerbi
Sub-service: pbi-explore
GitHub Login: @mihart
Microsoft Alias: mihart

You're right! Thanks for pointing that out, and it's a quick fix. I'll get that fixed and updated - should be in the live doc set in a day or so. Thanks again @SteveHills.


Change to extract abstract pager and convert Pager to a module #16

This extracts a common TTY::Pager::Abstract interface that requires overriding
write, puts and close methods. Changed Pager to be a module since it no longer
requires instantiation. Kept the #new method as a factory for the null, basic
and system pager instances. Made the #select_pager private to protect against
future changes. Only the top-level #page method remains as a convenience for
instance paging.
Describe the change
Improves maintainability.
Why are we doing this?
Better abstractions and maintenance
Benefits
Clearer codebase.
Drawbacks
Not that I can see.
Requirements
Put an X between brackets on each line if you have done the item:
[x] Tests written & passing locally?
[x] Code style checked?
[] Rebased with master branch?
[x] Documentation updated?

Coverage increased (+2.0%) to 92.271% when pulling 060d96954d00b906f830b8d19d188758d5555d51 on abstract-pager into 7f9173f2acc367f751bf95b5cfdf31d061a13f73 on master.

@AndrewRadev What do you think?


[bug] enforce/batch_enforce API no work

hi team,
I found the enforce/batch_enforce function of SDK can't work with the new version of Casdoor. it is because the casdoor enforce API's parameter format has been changed.
API descript:
https://casdoor.org/docs/permission/exposed-casbin-apis/#enforce
code:
https://github.com/casdoor/casdoor-python-sdk/blob/master/src/casdoor/main.py#L250-L336
https://github.com/casdoor/casdoor-python-sdk/blob/master/src/casdoor/async_main.py#L262-L331

@jump2cn does casdoor-go-sdk work?

@jump2cn any update?


RuboCop exception handling of autocorrecting cop has multiple issues

While working on #7868, I realized that RuboCop's exception handling of autocorrecting cop has issues:

some exceptions may be completely ignored:

# my_cop.rb:
module Custom
  class MyCop < RuboCop::Cop::Cop
    MSG = "let's be positive"

    def on_send(node)
      if node.method_name == :-
        add_offense(node)
        raise 'oh!'  # <<< Completely ignored!
      end
    end

    def autocorrect(node)
      -> (corrector) do
        corrector.replace(node.loc.selector, '+')
      end
    end
  end
end

> echo "2-2" | rubocop -r ./my_cop.rb --only Custom -a -s "test"

Inspecting 1 file
C

Offenses:

test:1:1: C: [Corrected] Custom/MyCop: let's be positive
2-2
^^^

1 file inspected, 1 offense detected, 1 offense corrected
====================
2+2

as noted in #7868, some exceptions will crash rubocop (move the raise inside the -> in the example above)

There are 3 specs for this handling in team_spec, but 1 is redundant with the previous spec, and there are quite a few cases missing and don't test the interaction with Runner. For example this spec fails if we add let(:options) { { auto_correct: true } }.
I don't intend on cleaning this up before #7868 is completed. It should be easier to deal with too as the current autocorrection timing is quite intricate.

I believe this to be fixed in #7868


Feat/set in discord false controller

Create a controller to consume previously created services and set in-discord of a user to false
After a discussion with ankush over the standup call on 15 June 2023 at 6:00 am IST we can skip 5xx tests for this PR.

Create a controller to consume previously created services and set in-discord of a user to false
After a discussion with ankush over the standup call on 15 June 2023 at 6:00 am IST we can skip 5xx tests for this PR.

What is the reason to skip 5xx for tests?
Please be more descriptive this helps other reviewers.

What is the reason to skip 5xx for tests?
Please be more descriptive this helps other reviewers.

The reason to skip 5xx tests is that 5xx only occurs when firebase fails to work, and I was not able to emulate that thus I asked Ankush and he told I cand send the PR and get it merged and send the pr for 5xx errors in future.


Remove *.svg - duplicate of 'common', should be 'binary'

*.svg is declared in Common.gitattributes
Type are also different between Java vs. Common.

Good catch, they are different because they were changed in a recent pull request. Thanks!


v0.4.17 Atrial fibrillation haveBleedingRiskFeature, haveAtrialFibExclusions and haveAtrialFibExclReq2 mutiple-choice handling of none is not correct

For the Atrial fibrillation trial, the criteria for the questions haveBleedingRiskFeature, haveAtrialFibExclusions and haveAtrialFibExclReq2 is not correct. Please fix it.

Should be fixed with 97b0e67cd1264756836d3596ee40fe0dac493eac but there is no "None of the above" option.


Add row key

Add the ability to set the React key of rows by specifying which column value to use for the key. This will allow users to either
use the value from any column field or to create an 'invisible' property on the rowData object that isn't used in a column to serve as the unique key.
I've been using this package recently and it's been great but I've ran into huge troubles around row deletion due to the key for rows defaulting to being the index of the row. This would work by adding an extra rowKey prop to the sheet which you can feed the string name of whichever column/property you want. For adding new rows all you need to do is create the value inside the function pased to createRow and keys should become consistent allowing for more reliable deletion of rows.

Thanks for the feedback! My first time doing any open source contributions :)
All comments should (hopefully) now be resolved. It's a little beefier with some more type checking and I still couldn't find a way to remove the typecasting completely, but it's done in a much more sensible place now at least.

No probs on the second one you're absolute right I just wasn't thinking! Regarding the first one, not sure exactly how to do that from 4 commits back in time. Is there a process with git I'm not aware of? Wasn't sure if it'd be easier to just make a new commit overall minus the package-lock changes to begin with.
As a follow up question. What should I have done in the first place? Normally comitting package-lock.json is fine so I went through with it here even with a ton of changes.

There might be other ways, but I would do it like this:

Create another branch from master
Cherry-pick the first commit
Checkout only package.lock.json from master with git checkout master package.lock.json, that should change the file back to the value from master
Amend commit with git commit --amend, that should remove the change from your commit
Cherry-pick the other commits
Push force to this origin branch

Yeah not sure about it really. Still seemed to create that merge error when I force pushed up. Had a last ditch effort at trying to resolve it by just pasting in the whole file as is now but it's created more changes inevitably. I'm fine with making a quick new PR without the package-lock if you don't mind the extra PR.

No problem, you can create a new PR if that makes it simpler, just make sure to update npm to its latest version ;)


Cant start .exe on Server

Hey,
for some reason i have a problem on servers to start the .exe.
Im getting this error.

Are there any logs i can share?

Depending on the framework it was built it might actually required .NET Framework 3.5. At least on some of my servers that didn't had .NET 3.5 it failed with the same.


Remap parts of Japanese layout to English layout

Thank you for karabiner. I have a macbook pro with a Japanese keyboard. I want to use the keyboard as an English keyboard, because that is what I have as an external keyboard. I can do a bulk mapping by simply changing to US layout in system preferences, so even though the keys are mis-labeled, I can find what I need.
However, there are some keys that are now unused, since the driver does not recognize them.
Can I use karabiner to remap these keys? I mean, the yen key to the left of backspace, and the ro/underbar key to the left of right shift. Or are these now "dead" to the driver?
Alternatively, if I keep the keyboard in Japanese mode, is there a way to do a remapping of individual keys, one by one, like:
shift semicolon -> colon
colon -> single quote
shift colon -> double quote
etc...? It seems like things might loop in an unexpected way.
I hope someone might be able to shed some light on this. Thanks!

@tekezo さま、お世話になります。コグレーです。英語はまったく完璧かもしれないが、日本語で頑張ってみます。結局、日本語配列のmacbook proで、外付けキーボードが英語のもので、内蔵のボードを英語に切り替えれば、「死んでいる」キーが出る。例えば、ろとか、¥とか。英語のまま、この死んでいるキーをkarabinerで、どうにかなるものでしょうか？それとも、日本語設定にし、一個一個リマップする必要があるのでしょうか？ご教授ください。よろしくお願いいたします。


spotx, spoty tuple --> coords

Please check if this works in earlier versions of Python (including the latest Python 2). Lines 227 and 228 might be a problem.
If it doesn't work, I'll refactor check_move_legal and board_to_step to make it work.

It should works as I have seen some scripts using * keywork in python 2 and this is a great python technique, Thanks!

Hi, I guess this is some problem using * keywords.
I run this version in my python 2 and it returns

Memoir@Purelove ~/d/p/p/g/t/ticky> python ticky.py                       master
File "ticky.py", line 227
if coords and check_move_legal(*coords, board) and not game_over:
SyntaxError: only named arguments may follow *expression

I think the NONE type is the problem.

Based on this:

SyntaxError: only named arguments may follow *expression

Try to change the calling of check_move_legal to: check_move_legal(*coords, board=board), and report back if it works.

I change it in this way and it works now.

if coords and check_move_legal(*([coords[0],coords[1]]+[board])) and not game_over:

This is a stackoverflow questions regarding this problem. This (*coords, board) should work for python 3.5 and not python 2. We actually need to concat them into a single list or tuple to pass to a function.
http://stackoverflow.com/questions/12720450/unpacking-arguments-only-named-arguments-may-follow-expression

I'll make slight refactoring to make it both elegant and working, give me couple of minutes.


Stepchain: support trident configurations

Stepchain should support parameters like input step and input module such that we can chain the steps as we want. For instance,

Step1 produces GEN-SIM-RECO
Step2 reads the GEN-SIM-RECO from Step1 and produces AODSIM
Step3 also reads the GEN-SIM-RECO from Step1 and produces AODSIM
Differences between Step2 and Step3 could be a != release, or GT or config or etc

Setting milestone to January, feel free to change it in case I'm wrong.

This will allow each intermediate output to be stored, correct?

That's a different, probably simpler, issue.

Fixed by #6589


Bluetooth: TBS: incoming and originating call on GTBS

It was not allowed to simulate incoming calls or originate calls on GTBS service instance. This change let these actions also be performed on GTBS. It can be seen as preparation for having only GTBS available and no instance of TBS.

Hmm, that has actually been done on purpose.
Since GTBS (in Zephyr at least) represent all the other TBS instances, it cannot have its own calls.
Of course in the case where there are no other TBS instances, it should be allowed to.
I'm not against this change, but it needs to done differently, if there are TBS instances instantiated.
It should do something similar to
		if (is_gtbs) {
			tbs = lookup_inst_by_uri_scheme(ccp->originate.uri, uri_len);
			if (tbs == NULL) {
				/* TODO: Couldn't find fitting TBS instance;
				 * use the first. If we want to be
				 * restrictive about URIs, return
				 * Invalid Caller ID instead
				 */
				tbs = &svc_insts[0];
			}
		}

Ok, I see. I will have a deeper look on this next week and implement the changes differently. Shall I set this PR as draft until I am finished?

Ok, I see. I will have a deeper look on this next week and implement the changes differently. Shall I set this PR as draft until I am finished?

I'd prefer that :)

I realized that this is not as easy to implement as I thought. It would require changes on many locations, so I would propose this to be implemented together with #41738 as by then it would be possible to really only have the GTBS instance running, without any TBS instance.


fix(cli): Improve errors related to cargo script

What does this PR try to resolve?
Fixes #13332
How should we test and review this PR?
See tests in last commit to see how this changes error messages.
This is a lot of duplication with minor tweaking that will go away on stabilization
Additional information

:pushpin: Commit 51b1200572645664d62a9bd19c6b09e85925d866 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 51b1200572645664d62a9bd19c6b09e85925d866 with merge f4bafa7f9180224e970185eec082dcb828542d02...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing f4bafa7f9180224e970185eec082dcb828542d02 to master...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing f4bafa7f9180224e970185eec082dcb828542d02 to master...

:eyes: Test was successful, but fast-forwarding failed: 422 Changes must be made through a pull request.


Lock consistency violation in itemlist_formaction

As reported in https://github.com/newsboat/newsboat/issues/254#issuecomment-415651046, Infer static analyzer found an issue with accesses to invalidated and invalidation_mode fields:

src/itemlist_formaction.cpp:821: error: LOCK_CONSISTENCY_VIOLATION
Read/Write race. Non-private method newsboat::itemlist_formaction_do_update_visible_items reads without synchronization from this.invalidated. Potentially races with write in method prepare.

src/itemlist_formaction.cpp:821: error: LOCK_CONSISTENCY_VIOLATION
Read/Write race. Non-private method newsboat::itemlist_formaction_do_update_visible_items reads without synchronization from this.invalidation_mode. Potentially races with write in method prepare.

src/itemlist_formaction.cpp:706: error: LOCK_CONSISTENCY_VIOLATION
Read/Write race. Non-private method newsboat::itemlist_formaction_finished_qna indirectly reads without synchronization from this.invalidated. Potentially races with write in method prepare.

src/itemlist_formaction.cpp:995: error: LOCK_CONSISTENCY_VIOLATION
Read/Write race. Non-private method newsboat::itemlist_formaction_init indirectly reads without synchronization from this.invalidated. Potentially races with write in method prepare.

src/itemlist_formaction.cpp:1346: error: LOCK_CONSISTENCY_VIOLATION
Read/Write race. Non-private method newsboat::itemlist_formaction_set_feed indirectly reads without synchronization from this.invalidated. Potentially races with write in method prepare.

The first two items seem to be easily fixable by turning redraw_mtx into a resursive_mutex, but the rest need a deeper investigation to find where accesses happen, and then protect those with redraw_mtx as well. Or maybe introduce a separate mutex just for invalidation stuff.

I wonder if this is related to some of the memory corruption bugs I keep running into. I built 2.15 with debug info and got some Valgrind errors that look suspiciously similar. "Invalid write" to location that was free'd by some other location. related to ItemListFormAction::prepare.
newboat-valgrind2.txt

Looks like invalid writes are happening in ListFormatter, but ItemListFormAction only uses it inside prepare(), which is protected by a mutex. So not sure how related those are to Infer warnings. Thanks for checking with Valgrind anyway.


Disable staticcheck golangci-lint

This check has been failing master silently for several days.
https://github.com/zmap/zlint/actions/runs/651345041
Essentially, it doesn't seem to like that we import crypto/dsa at any point.

SA1019: package crypto/dsa is deprecated: DSA is a legacy algorithm, and modern alternatives such as Ed25519 (implemented by package crypto/ed25519) should be used instead. Keys with 1024-bit moduli (L1024N160 parameters) are cryptographically weak, while bigger keys are not widely supported. Note that FIPS 186-5 no longer approves DSA for signature generation. (staticcheck)

This might actually be something that we want to resolve rather than commenting out. If the DSA package is deprecated, it raises the question, "When will the package be removed?" and our code break. Does it make more sense to instead move this to ZCrypto package?

@zakird I reckon that the question of when we can remove our dependence on this package is largely predicated on the question of when we can remove any lints that might ask any questions about DSA (including lints that insist that it not be algorithm of choice of a particular cert).
But really, when this error says that package crypto/dsa is deprecated I sort of reckon they don't mean "this package will get straight up deleted". I'm reading that as, "please please don't use this, please use something better - I am begging you".

This check has been failing master silently for several days.

@christopher-henderson Nice catch! It might make sense to pair the cron builds with an email notification for failures to make this more obvious in the future.

Does it make more sense to instead move this to ZCrypto package?

@zakird agreed, this seems like the best longer term fix :thumbsup:

I'd be amenable to that. Would that essentially mean vendoring the dsa package under ZCrypto and changing all of these imports to zcrypro/dsa?

Yes