[WIP] Sparse matmul

This PR aims to support sparse matmul for graph convolution network.
A function named sparse_matmul is added with this PR. The use case of this function is "C (dense) = A (sparse) * B (dense)" and you can use it as follows:
c = F.sparse_matmul(b, a_value, a_row_index, a_col_index)

A sparse matrix storage format adopted by this PR is simple one which keeps "value", "row index" and "column index" of sparse matrix separately. This is mainly because it is easy to implement custom cupy kernel for this format and you can use the kernel both for forward and backward. However, I have to say that I have no idea on efficient numpy implementation of sparse matmul with this format. In other word, you can use this function only on GPU now. I would like to ask numpy magician regarding the implementation with numpy.
Furthermore, sparse version of GCN is implemented as an use case of sparse matmul. So, you can test it with tox21 dataset. Any feedback would be appreciated.

Thank you for your quick implementation.
I guess your implementation follows the "COO" sparse matrix format.
I  guess scipy.sparse is usually used for CPU version of sparse matrix handling. But it does not support 3-dim sparse matrix which includes minibatch axis, so naive extension seems difficult.
Let us consider & review how to deal with sparse matrix implementation for graph convolution.
One question: How is the speed difference between GCN and SparseGCN?
Is it faster to use SparseGCN??

I was also investigating sparse matrix, and testing performance of cupy.sparse.
Let me just share it.
"""
http://ohke.hateblo.jp/entry/2018/01/07/230000
https://docs-cupy.chainer.org/en/stable/reference/sparse.html

Please install library
pip install git+https://github.com/corochann/chainerex
"""
# import numpy as xp
# from scipy import sparse
import numpy as np
import cupy as xp
from cupy import sparse
from chainer import cuda

def allclose(a, b):
    return np.allclose(cuda.to_cpu(a), cuda.to_cpu(b))

# 1000×1000の疎行列の作成
# dimensions = 50000  # OutOfMemory error
dimensions = 10000
# dimensions = 10000
# dimensions = 1000
# dimensions = 100
# sparse_matrix = xp.zeros((dimensions, dimensions), dtype=xp.int64)
sparse_matrix = xp.zeros((dimensions, dimensions), dtype=xp.float32)

# non_zero_elements = 10000  # 非ゼロ要素は全体の1%
# non_zero_elements = 1000  # 非ゼロ要素は全体の0.1%
non_zero_elements = 100  # 非ゼロ要素は全体の0.01%
count = 0
rows = xp.random.randint(0, dimensions, 20000)
columns = xp.random.randint(0, dimensions, 20000)
values = xp.random.randint(0, 10, 20000)
for i in range(0, 20000):
    if sparse_matrix[rows[i], columns[i]] == 0:
        sparse_matrix[rows[i], columns[i]] = values[i]
        count += 1
        if count == non_zero_elements:
            break

# sparse_lil = sparse.lil_matrix(sparse_matrix)
sparse_csr = sparse.csr_matrix(sparse_matrix)

import chainerex.utils as cl
tm = cl.TimeMeasure()
# 疎行列の内積速度の計測

tm.update('start')
a = xp.dot(sparse_matrix, sparse_matrix)
tm.update('xp dot xp-> {} {}'.format(type(a), a.shape))
# b = sparse_lil * sparse_lil
# tm.update('lil dot -> {} {}'.format(type(b), b.shape))
c = sparse_csr * sparse_csr
tm.update('csr * csr -> {} {}'.format(type(c), c.shape))
cdot = sparse_csr.dot(sparse_csr)
tm.update('csr dot csr -> {} {}'.format(type(cdot), cdot.shape))
d = sparse_csr.dot(sparse_matrix)
tm.update('csr dot xp -> {} {}'.format(type(d), d.shape))
e = sparse_csr.dot(sparse_matrix.T)
tm.update('csr dot xp.T -> {} {}'.format(type(e), e.shape))

its result is:
$ python sparse_matrix_dot_time_cupy.py
[TimeMeasure] start: 0.000002 sec
[TimeMeasure] xp dot xp -> <class 'cupy.core.core.ndarray'> (10000, 10000): 0.000242 sec
[TimeMeasure] csr * csr -> <class 'cupy.sparse.csr.csr_matrix'> (10000, 10000): 0.225117 sec
[TimeMeasure] csr dot csr -> <class 'cupy.sparse.csr.csr_matrix'> (10000, 10000): 0.000281 sec
[TimeMeasure] csr dot xp -> <class 'cupy.core.core.ndarray'> (10000, 10000): 0.002072 sec
[TimeMeasure] csr dot xp.T -> <class 'cupy.core.core.ndarray'> (10000, 10000): 0.000165 sec

It shows that the improvement is not so huge (0.000242 sec -> 0.000165 sec), and only when memory alignment is optimal (e.g., csr dot xp is slower than xp dot xp).

I was considering a way to utilize cupy.sparse so that we can get benefit of cuSPARSE.
In that case, we may need to use csr format and it seems not easy to use matmul.
Only dot operation is supported in sparse matrix.
But when I look at the picture uploaded on here,
https://github.com/tkipf/gcn#graph-classification
it seems it is possible to use sparse matrix by concatenating along minibatch axis to represent nodes (atoms) of each molecules.

Thank you for your comment.
Re: performance of GCN and SparseGCN, I think we should measure the performance using various size of data-sets. So, I made some synthetic data-sets named Tox21xN in which number of atoms of each molecule is N times as many as original Tox21. The measured results are as follows.
[Elapsed time (sec)]
            GCN     SparseGCN
Tox21       41.4       37.2
Tox21x2     46.7       39.5
Tox21x4     91.9       41.5
Tox21x8    274         41.4
Tox21x16     NA        45.6
Tox21x32     NA        51.6
Tox21x64     NA        75.7
Tox21x128    NA       125

Batch size: 128
Epochs: 20
Chainer: 3.3.0
CuPy: 2.3.0
CUDA: 9.0
cuDNN: 7.0
Python: 3.6.1 (pyenv anaconda3-4.4.0)
GPU: Tesla V100
CPU: Intel Xeon E5-2698

The result says that sparse matmul is kind of "must to have" when graph size is big.

Thank you for experiment & quick response.
It seems quite promising, we should consider sparse matmul functionality.
Let us discuss about CPU version implementation & how to deal with API.

I've updated the branch.

The latest master (a5e8f9e70627ae733b45749f92296b0cb67884a4) is based to reflect some changes.
sparse_matmul implementation submitted to Chainer (https://github.com/chainer/chainer/pull/4397) is adopted.
The name of model "SparseGCN" is changed to "SparseRSGCN".
The option padding in concat_mol is appropriately set when "SarseRSGCN" is used.

Now, sparse_matmul is available on CPU too.

I have an idea of efficient sparse matmul computation on CPU.
Currently I guess you iterate over minibatch axis, since sparse matmul is only applicable to 2-dim matrix.
https://github.com/pfnet-research/chainer-chemistry/pull/90/files#diff-3a3f705484aa44f5789ea2dcb039de9eR116
Then, how about creating a big sparse coo matrix whose row_index & col_index is shifted according to the batch index?
Please see this figure for intuitive understanding.
https://user-images.githubusercontent.com/7347296/34198790-eb5bec96-e56b-11e7-90d5-157800e042de.png
So let's say you want to calculate F.sparse_matmul(B (sparse), A (dense)), we can reshape A to the following, and reconstruct sparse matrix B' in following way,
A (minibatch, num_nodes, ch) --> A' (minibatch * atoms, ch)
B (minibatch, num_nodes, num_nodes) --> B' (minibatch * atoms, minibatch * atoms)
Then we can calculate to obtain
C' = F.sparse_matmul(B', A') to obtain C' (minibatch * num_nodes, ch), finally we can reshape it to obtain C (minibatch, num_nodes, ch).
Since B' is a big sparse matrix, we only need to calculate sparse_matmul once.
What do you think of this idea (I am not testing the function, I may be saying incorrect or impossible idea...)?

Basically, I think it is good idea to concatenate many small sparse matrices together and make a single big sparse matrix and run sparse_matmul once. If the sparse matrices concatenation is fast enough compared with computation of sparse_matmul, it will be good not only for CPU but also for GPU. Probably, it is also good for computation of gradients of dense matrix (more exactly, batch of dense matrices) because the single big sparse matrix can be re-used for that computation (when it is COO format).
My concern now is how to compute gradients of sparse matrices. It needs to perform "g_A (sparse) = g_C (dense) * B (dense)" (assuming you perform "C (dense) = A (sparse) * B (dense)" in forward path). For GPU, I wrote custom kernel with CuPy for this computation. But, for CPU, there is no functions in SciPy and Numpy to perform this computation, AFAIK. Thus, I first perform "g_A (dense) = g_C (dense) * B (dense)" using matmul and then make "g_A (sparse)" by picking up necessary elements from "g_A (dense)"
(please refer to https://github.com/anaruse/chainer-chemistry/blob/fc3f184dc589564197578834721e5d0498d0b194/chainer_chemistry/functions/sparse_matmul.py#L310). As you can see, we can not take this approach for single big sparse matrix, because shape of "g_A (dense)" becomes (minibatch * num_nodes, minibatch * num_nodes) and it is too large. Maybe we can iterate over batch axis for this, but it means we have to keep two types of sparse matrices: batched sparse matrix and concatenated sparse matrix. It seems to me that it is not excellent solution.
What would you think on this?
BTW, computation of gradients of sparse matrices is not necessary for RSGCN. If we only think about networks like RSGCN, we can take "sparse matrices concatenation" approach.

Ah, I see... situation is different between forward and backward...

If we only think about networks like RSGCN, we can take "sparse matrices concatenation" approach.
I also understand that we may use different approach when "only forward" is necessary or "forward & backward" is necessary for optimization...

So far, for graph convolution network, I think we only need "forward" computation for sparse matrix A and we only need gradient for dense matrix B most of the time.

Although it might have looked as if I was rather negative to your idea, I think it is worth a try.  I will first implement it in the way that it concatenates sparse matrices on demand basis.

Thank you very much for discussion. I think we should decide our direction/implementation design.
So sorry but please let us have time to consider/discuss. Please prioritize your other work until then.

Sorry for the review delayed.
We would like to proceed review & merge (maybe as experimental feature as first stage).
If you have updates on your code, please let us know.

For now, I've updated the latest code on the branch. The following features were added.

Fused sparse matmul:  You can test this feature using option --flatten in tox21 example. Note that you need to use sparse_rsgcn model to test this feature. When you select this mode, multiple sparse matrices are first to be fused on CPU then transferred to GPU. Please see concat_sparse_rsgcn for details.
Tox21xN dataset:  This is only for performance measurement. you can specify N with option --multiplier in tox21 example.

Several conflicts to master branch have been resolved.

I looked through your code, currently following implementations are done
・Sparse matrix Calculation
１．sparse_matmul with fuse
２．sparse_matmul with Minibatch axis
in different kernel
・converter for Fuse sparse matrix
・Artificial big Tox21 dataset
・SparseRSGCN implementation
Since there are many things to review, we would like to merge it as new branch experimental_sparse to brush up & discuss implementation. Is this direction ok for you? @anaruse

Yes, I agree with you about how to proceed the review.

Let me merge this PR on experimental_sparse branch.


Travis CI notifications in Hipchat

Travis CI
This adds travis notifications in our Hipchat room.  It will only notify when the build status changes.
I also moved the database migrate from scripts to before_script.  The reason for this is because travis will collapse the output from anything run in before_script.  Anything run in scripts won't be collapsed, and the migrate task is very verbose, leading to builds that are hard to read.
Spec fix
I also increased the time given for allowing Solr to start, since the original 10 seconds sometimes was not enough time for Travis to start Solr.

:+1:

:+1:  :+1:  :+1:  :+1:  :+1:  :+1:

:100: x :+1:


net: builders failing TestLookupDotsWithRemoteSource and TestLookupGoogleSRV due to missing host for _xmpp-server._tcp.google.com

#!watchflakes
post <- pkg == "net" && test == "TestLookupGoogleSRV"

--- FAIL: TestLookupDotsWithRemoteSource (0.32s)
    lookup_test.go:708: LookupSRV(xmpp-server, tcp, google.com): lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host (mode=go)
    lookup_test.go:708: LookupSRV(xmpp-server, tcp, google.com): lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host (mode=cgo)
--- FAIL: TestLookupGoogleSRV (36.06s)
    lookup_test.go:93: backoff 1s after failure lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host
    lookup_test.go:93: backoff 5s after failure lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host
    lookup_test.go:93: backoff 30s after failure lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host
    lookup_test.go:99: lookup _xmpp-server._tcp.google.com on <IP_ADDRESS>:53: no such host
FAIL
FAIL	net	39.011s

greplogs -l -e xmpp-server --since=2022-11-01
2022-11-11T16:04:21-fcd14bd/darwin-amd64-11-aws
2022-11-11T16:04:21-fcd14bd/illumos-amd64
2022-11-11T16:04:21-fcd14bd/linux-386
2022-11-11T16:04:21-fcd14bd/linux-386-buster
2022-11-11T16:04:21-fcd14bd/linux-amd64-goamd64v3
2022-11-11T16:04:21-fcd14bd/linux-amd64-staticlockranking
2022-11-11T16:04:21-fcd14bd/linux-arm64
2022-11-11T16:04:21-fcd14bd/linux-arm64-aws
2022-11-11T16:04:21-fcd14bd/netbsd-386-9_0
2022-11-11T16:04:21-fcd14bd/openbsd-386-68
2022-11-11T16:04:21-fcd14bd/windows-386-2012
2022-11-11T16:04:21-fcd14bd/windows-amd64-2008
2022-11-11T16:04:21-fcd14bd/windows-amd64-2016-newcc
2022-11-11T16:04:21-fcd14bd/windows-amd64-newcc-race
2022-11-11T16:04:21-fcd14bd/windows-arm64-10
2022-11-11T16:04:21-fcd14bd/windows-arm64-11
2022-11-11T14:51:14-fffda6b/linux-386-sid
2022-11-10T18:46:00-fbf763f/plan9-arm

(attn @ianlancetaylor @neild; CC @mknyszek)

Filed #56708 to follow-up.

Needs backporting?

I think so, yeah.

@gopherbot Please backport this issue to Go 1.18 and Go 1.19. It's causing widespread builder failures.

@mknyszek @bcmills  should this be backported to boring branches? Tests are failing there too and will hit this issue when cutting a new release.

This issue was fixed in Go1.20 milestone and frozen, with Soon label. The new failure mode is different, and watchflakes probably shouldn't reopen frozen issues (filed #67232 for that). I'll take it out of the Test Flakes project to avoid that happening again.


Add JWT token auth provider based on djangorestframework-simplejwt

As proposed/discussed in #34

@barseghyanartur thanks for this pull request.
Code looks good but I do not have an easy way to test. Do you think it would be easiest to add to jwt to example app?

@barseghyanartur thanks for this pull request.
Code looks good but I do not have an easy way to test. Do you think it would be easiest to add to jwt to example app?

Great! Thanks for checking. I thought tests should be sufficient, but sure - I will add a working example (later today/tomorrow) and let you know.

@bmihelac:
Example app is updated. Please, check.

@barseghyanartur merged, thanks!

@bmihelac:
Thanks! Could a npmjs release be expected one day soon? :)


helm install fails waiting for response

The runbook step to execute helm install using
helm install releasename local-development/spm
is failing on a timeout.  There doesn't appear to be detailed logs I can find.  Please see the attached screenshot of the dashboard.  it shows a timeout event on the customsql job.   However, when I Execute a curl command from the command line I get a response immediately.
[ibmadmin@localserver helm-charts]$ curl http://minikube.local:5000/v2
Moved Permanently.
If I attempt to install again, I get this error message.
[ibmadmin@localserver helm-charts]$ helm install spm-v1 local-development/spm
Error: cannot re-use a name that is still in use
Please advise.

Hi,
Please confirm that the Minikube VM has been configured as per our guide, specifically setting the --insecure-registry and --memory flags?
minikube start --vm-driver=virtualbox --cpus 4 --memory 8G \
         --insecure-registry "<IP_ADDRESS>/16" --disk-size='30G' --kubernetes-version v1.18.6

From your screenshot, it appears that the pod is attempting to reach the Minikube registry using HTTPS, whereas it should be using HTTP (i.e. insecure registry).
If the IP of your Minikube environment was generated in a different subnet, you may need to adjust your start command.
Run minikube ip to check if your environment is using the <IP_ADDRESS>/16 range (e.g. <IP_ADDRESS>). If not, you will need to re-create the minikube cluster setting the --insecure-registry flag accordingly.

Good catch on the https.  Here are the commands I used.  I used 6 GB because I'm running this inside a VM where the whole VM is allocated 8 GB.
[ibmadmin@localserver helm-charts]$ minikube start --vm-driver=docker --cpus 2 --memory 6G --insecure-registry "<IP_ADDRESS>/16" --disk-size='15G' --kubernetes-version v1.18.3
[ibmadmin@localserver helm-charts]$ minikube addons enable registry
[ibmadmin@localserver helm-charts]$ minikube ip
<IP_ADDRESS>

[ibmadmin@localserver helm-charts]$ sudo echo -e "$(minikube ip)\tminikube.local" | sudo tee -a /etc/hosts
<IP_ADDRESS>	minikube.local

I deleted the cluster by executing minikube delete and then recreated it using these commands above.  However I get the same result you see in the screenshot.
Is there another setting for the https issue?

I only notice now that you're also using the docker driver for Minikube, which appears to be a contributing factor to the documented commands not working quite as expected - the runbook was only validated with Minikube using VirtualBox and VMware Fusion drivers.
I was able to reproduce your issue using an image from the Minikube docs:
docker pull k8s.gcr.io/echoserver:1.4
docker tag k8s.gcr.io/echoserver:1.4 minikube.local:5000/echoserver:1.4
docker push minikube.local:5000/echoserver:1.4

kubectl create deployment hello-node-1 --image=minikube.local:5000/echoserver:1.4
kubectl create deployment hello-node-2 --image=localhost:5000/echoserver:1.4

The pod for hello-node-1 gets stuck with an ImagePullBackOff error.
The pod for hello-node-2 is created successfully.
The implementation of the docker driver isn't able to resolve that test domain in some instances, and when it does, it's in a different internal IP range that what is specified in the --insecure-registry flag.
Please update your override values file to refer to localhost:5000 instead of minikube.local:5000 for pulling images
global:
  images:
    registry: localhost:5000

Thanks Andrey.  That worked.  I am able to pull the image now.
However, it hit another error related to DB2.  I'll open another issue on that one.

Great, thanks for confirming. I'll close this since this specific problem is resolved.


Library import<PHONE_NUMBER>

PR was created by rightlib sync script

checks_integrated failed, disabling future checks

Check checks_integrated failed. All future check are suspended, please remove the {self.pr_label_fail} label to enable checks.

Check checks_integrated failed. All future check are suspended, please remove the rightlib-fail label to enable checks.

The PR was successfully merged into main using workflow


Correct flag indirect selection.

Contributions

[X] I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates
https://docs.getdbt.com/reference/global-configs/indirect-selection
What part(s) of the page would you like to see updated?
Hi. At the beginning of the docs, this text is displayed:
"Use the --indirect_selection flag to dbt test or dbt build to configure which tests to run for the nodes you specify. You can set this as a CLI flag or an environment variable. In dbt Core, you can also configure user configurations in YAML selectors or in the flags: block of dbt_project.yml, which sets project-level flags."
It should be replaced by this:
"Use the --indirect-selection flag to dbt test or dbt build to configure which tests to run for the nodes you specify. You can set this as a CLI flag or an environment variable. In dbt Core, you can also configure user configurations in YAML selectors or in the flags: block of dbt_project.yml, which sets project-level flags."
Also it would be a good idea to add the "edit" option to the website.
Regards.
IL.
Additional information
No response

Thanks, @ialdg Fixing it.

Also it would be a good idea to add the "edit" option to the website.

All of our page do have an "edit this page" option at the bottom. It's a little difficult to find, we're working on ways to make it better


Admin dashboard update

New admin dashboard UI ready for logic

The depicted scenario shouldn't occur. To address this issue, consider either hiding the "Rodyti ištrintus pranešimus" option when dumps are selected or disabling it. This adjustment will enhance the user experience and prevent potential confusion.


NEXT RELEASE MUST BE V5.0.0

This package follows semver. This promise was made initially, and cannot be broken.

I've reverted the tag of v4.5.1.

Thanks @GrahamCampbell for this. Will keep this pattern in mind going forward

Also, note there's a branch alias in the composer.json. It tells you the current master represents dev on 5.0.x. In the future, you'll wanna bump that alias as dev represents later versions.


update rules

About the PR
security will finally be in check
Changelog

:cl:

tweak: The game rules have been updated to bring the server to Low Roleplay standards. Go read 'em!

Merging already because we've already discussed and agreed on them


ruby: bad interpreter: No such file or directory

$ terraforming r53r --tfstate
/usr/local/bin/terraforming: /usr/local/Cellar/terraforming/0.18.0/libexec/bin/terraforming: /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby: bad interpreter: No such file or directory
/usr/local/bin/terraforming: line 2: /usr/local/Cellar/terraforming/0.18.0/libexec/bin/terraforming: Undefined error: 0

ProductName:	Mac OS X
ProductVersion:	10.15.1
BuildVersion:	19B68f
$ brew --version
Homebrew 2.1.14
Homebrew/homebrew-core (git revision 3592; last commit 2019-10-15)
Homebrew/homebrew-cask (git revision 6746e7; last commit 2019-10-15)

@chenrui333 not sure you can help here?

I have this issue as well:
ProductName: Mac OS X
ProductVersion: 10.15
BuildVersion:  19A602
brew --version
Homebrew 2.2.0
Homebrew/homebrew-core (git revision bd9f90; last commit 2019-12-06)
Homebrew/homebrew-cask (git revision b10c3; last commit 2019-12-06)
$ terraforming s3
/usr/local/bin/terraforming: /usr/local/Cellar/terraforming/0.18.0/libexec/bin/terraforming: /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby: bad interpreter: No such file or directory
/usr/local/bin/terraforming: line 2: /usr/local/Cellar/terraforming/0.18.0/libexec/bin/terraforming: Undefined error: 0

Sure, I can take a look.

I have just done the fresh installation and run the terraforming r53r --tfstate, it works well for me.

/System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby: bad interpreter: No such file or directory
that looks like a familiar issue to me.

can you run brew reinstall terraforming to see if it works?

I just performed a brew remove terraforming and then a subsequent brew install terraforming and it is working.
When I look in /usr/local/Cellar/terraforming/0.18.0/libexec/bin/terraforming, I see that the first line points to the 2.6 version of ruby: #!/System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby. I suspect this might break at some point. Would it make sense to always point to "Current"?
The contents of the /System/Library/Frameworks/Ruby.framework/Versions/ dir is:
$ ls -al /System/Library/Frameworks/Ruby.framework/Versions/
total 0
drwxr-xr-x  4 root  wheel  128 Oct 12 09:55 .
drwxr-xr-x  5 root  wheel  160 Oct 12 09:55 ..
drwxr-xr-x  6 root  wheel  192 Sep 29 16:29 2.6
lrwxr-xr-x  1 root  wheel    3 Oct 12 09:52 Current -> 2.6

/System/Library/Frameworks/Ruby.framework/Versions/ that is MacOS default ruby installation though (ruby 2.6 on Catalina OS). I think for every MacOS upgrade, it (homebrew stuff) might be broken.
I had the similar issue before and after remove ruby symlink (that is why I feel like I saw this earlier), it started working for me. :)

I have faced the similar issue due to ruby version not set. These are the steps that I have followed:
Check rbenv versions in your system by this command:
rbenv versions
if not then installed and set local or globally as per need by command:
rbenv local 2.6.6
the output of rbenv versions command like this-
* system (set by /Users/sakshijain/.rbenv/version)
  2.3.7
  2.4.0
  2.5.0
  2.6.3
  2.6.5
  2.6.6


chaijs 1.10 allows to write lint-friendly tests

be.ok; and others can now be written as be.ok();
Fixes https://github.com/switchfly/ember-cli-mocha/issues/6
More details https://github.com/chaijs/chai/pull/297

Note: this behavior has been reverted in Chai master, and is slated to go out in the next release. Additionally, using this lint-friendly syntax is incompatible with chai-as-promised.

o.O I was definitely not aware that this was rolled back thanks. It is still advertised on their homepage (for 2 months now) and I guess we were not the only ones converting our project ...

Same as https://github.com/switchfly/ember-mocha/pull/13#issuecomment-73361420

I'm using https://github.com/prodatakey/dirty-chai to deal with this.
I think the chai project is nuts for pushing a single style that forces people to disable a whole class of safety checks in their test code.


Content Creator: View edits across revisions

As a Content Creator I want to be able to see the edits across revisions so that I can see what changes have been made to a piece of content over time.
Feature: View edits across revisions
  As a Content Creator
  I want to be able to see and compare the edits across different revisions for a piece of content
  So that I know what changes were made to the content over time

  Scenario: Create DIFF
    Given {{thing}}
     When {{action}}
     Then {{result}}

{{feature}}

[ ] {{scenario}}

Will need to actually talk to users to see if this is a necessary feature for content creators. And how to design/implement. Could potentially be something that get a bit complicated.

@mkiang3051 - My thought is we allow users to only compare the current version to the previous version instead of letting users compare any version to any version, but that's just my opinion. Like you said, this needs to be flushed out after doing some user research.

@ineedsubstance Yea, I can see how it could get super complicated if they were able to compare any revision they want. And yep, we can flesh out as we do some user research.


metadata not being shown for a dandiset 26

https://gui.dandiarchive.org/#/dandiset/000026

https://github.com/dandisets/000026 says  1205 files, 863.7 GB, and shows that it is not in DANDI layout (may be that somehow matters)
console in the browser gives error:
instrument.js:108 TypeError: Cannot read property 'includes' of undefined
    at DandisetDetails.vue:267
    at Array.filter (<anonymous>)
    at o.contactName (DandisetDetails.vue:266)
    at en.get (vue.runtime.esm.js:4479)
    at en.evaluate (vue.runtime.esm.js:4584)
    at o.contactName (vue.runtime.esm.js:4836)
    at o.pi (DandisetDetails.vue?f4d8:1)
    at vue-composition-api.esm.js:1415
    at $t (vue-composition-api.esm.js:1375)
    at o.e.render (vue-composition-api.esm.js:1415)
    at o.t._render (vue.runtime.esm.js:3548)
    at o.n (vue.runtime.esm.js:4066)
    at en.get (vue.runtime.esm.js:4479)
    at new en (vue.runtime.esm.js:4468)
    at Ir (vue.runtime.esm.js:4073)
    at o.kn.$mount (vue.runtime.esm.js:8415)
    at init (vue.runtime.esm.js:3118)
    at p (vue.runtime.esm.js:5978)
    at d (vue.runtime.esm.js:5925)
    at b (vue.runtime.esm.js:6053)
    at d (vue.runtime.esm.js:5954)
    at b (vue.runtime.esm.js:6053)
    at d (vue.runtime.esm.js:5954)
    at b (vue.runtime.esm.js:6053)
    at d (vue.runtime.esm.js:5954)
    at O (vue.runtime.esm.js:6216)
    at A (vue.runtime.esm.js:6319)
    at o.__patch__ (vue.runtime.esm.js:6482)
    at o.Tr.t._update (vue.runtime.esm.js:3948)
    at o.n (vue.runtime.esm.js:4066)
    at en.get (vue.runtime.esm.js:4479)
    at en.run (vue.runtime.esm.js:4554)
    at Kr (vue.runtime.esm.js:4310)
    at Array.<anonymous> (vue.runtime.esm.js:1980)
    at ue (vue.runtime.esm.js:1906)
...

There's a couple things at play here. The first is that the schema was changed, so the GUI looks for contributor.roles, when it should be looking for contributor.roleName. However, that isn't occurring here, because this data is old, and still in the Girder system. The other case (that caused this) is if any contributor is missing the proper role field, it fails because it's using that field to try to find who to display as the "Contact" in the details panel. Both of these can be addressed short term, but this might be one of the friction points between the changing schema, and our GUI. At some point, the GUI has to make some assumptions about the data in order to display things properly, and this is one of them.

i'm going to close this since we have migrated.


Support other language, such as Chinese?

Hi whether this tool support other language, such as Chinese?

It does not support any other languages other then English and I don't think it ever will since is completely dependant on the 'nltk' library for natural language parsing.
Thanks, Jacob.


Add unique constraint to course-user relation

I'm not sure if this won't break something somewhere


12389 export form ODK XML for easier Collect import

Adds 2 new links:

export ODK XML for a particular form
export ODK XML for all published forms (zip file)

Docs: https://getnemo.readthedocs.io/en/latest/content/export/xml.html

specs are flappers, ready for review at your leisure!

This is awesome Kevin!!
I think I was expecting to be able to export the XML on the show/edit form page. Is there a reason you bundled them all up?

it's for both pages: you can export any individual form, or you can export in bulk.
currently, forms aren't rendered into XML until they're published, so that's why it's only for live/paused forms. it would require several additional things in order to render into XML sooner because it depends on some things such as having a version specified.


[6.0] [Completion] Only skip result builder expressions in the same builder

6.0 cherry-pick of #74219 + #74233

Explanation: Fixes code completion on values that depend on result builders being solved
Scope: Affects code completion around result builders
Issue: rdar://127154780, #72399
Risk: Low, causes us to avoid skipping result builder expressions in more cases, stress tester didn't show any performance-related failures
Testing: Added tests to test suite
Reviewer: Alex Hoppen

@swift-ci please test


[WFCORE-5598] Upgrade Undertow to 2.2.12.Final

Jira: https://issues.redhat.com/browse/WFCORE-5598
    Release Notes - Undertow - Version 2.2.12.Final

        Bug

[UNDERTOW-1969] -         Cannot create certificate from header on SE17

[UNDERTOW-1972] -         InMemorySessionManager can mistake PLACE_HOLDER_SESSION with a real session

    Release Notes - Undertow - Version 2.2.11.Final

        Sub-task

[UNDERTOW-1959] -         DefaultServletCachingListenerTestCase.testWelcomePages still fails sporadically with File was not refreshed

[UNDERTOW-1960] -         Address already in use in LoadBalancingProxy test cases

[UNDERTOW-1962] -         Prevent BindException (Address already in use) in all points that invoke Undertow.close() and XnioWorker.shutdown

        Bug

[UNDERTOW-1198] -         IPAddressAccessControlHandler does not recognize IPV6 loopback address ::1

[UNDERTOW-1532] -         PathResourceManager doesn't seem work properly with Windows shortened file names

[UNDERTOW-1568] -         AbstractFramedChannel.toString() is not synchronized

[UNDERTOW-1789] -         Bring back bumpTimeout in setMaxInactiveInterval of HttpSession

[UNDERTOW-1869] -         InMemorySessionManager Session Creation Not Thread Safe

[UNDERTOW-1910] -         AbstractFramedStreamSinkChannel.awaitWritable() never returns if peer fails to update HTTP2 window

[UNDERTOW-1917] -         Logic in try/finally blocks hides exceptions

[UNDERTOW-1931] -         Null pointers should not be dereferenced

[UNDERTOW-1932] -         ALPNHackSSLEngine field manipulation methods are not consistent.

[UNDERTOW-1933] -         Short-circuit logic for boolean

[UNDERTOW-1950] -         Fix NullPointerException in PreCompressedResourceSupplier

[UNDERTOW-1957] -         The undertow-servlet-jakartaee9 and undertow-websockets-jsr-jakartaee9 artifacts are not under dependency management

        Task

[UNDERTOW-1915] -         Remove unneccesary code

[UNDERTOW-1916] -         Review PathResourceManager exception handling

https://ci.wildfly.org/viewLog.html?buildId=274930&buildTypeId=WildFlyCore_PullRequest_WildFlyCoreFullIntegrationWindows failed due to the env problem discussed at https://wildfly.zulipchat.com/#narrow/stream/174184-wildfly-developers/topic/Clustering.20tests.20Failed.20to.20delete.20server.2Elog
I started another run as that problem prevented a lot of tests executing, but if it happens again and the other jobs look good I think we should ignore that job and get this merged.

@fl4via thanks


ImageNet Downloader - Dead Links

The imagenet downloader seems to use dead links.
Test Script:
from torchvision.datasets import ImageNet

train = ImageNet('/fast/ImageNet/train', split='train', download=True)
val = ImageNet('/fast/ImageNet/val', split='val', download=True)

Output:
0it [00:00, ?it/s]Downloading http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_devkit_t12.tar.gz to /fast/ImageNet/train/ILSVRC2012_devkit_t12.tar.gz
Traceback (most recent call last):
  File "/home/setepenre/work/mlbaselines/mlbaselines/datasets/__init__.py", line 4, in <module>
    train = ImageNet('/fast/ImageNet/train', split='train', download=True)
  File "/home/setepenre/miniconda3/lib/python3.7/site-packages/torchvision/datasets/imagenet.py", line 55, in __init__
    self.download()
  File "/home/setepenre/miniconda3/lib/python3.7/site-packages/torchvision/datasets/imagenet.py", line 75, in download
    md5=archive_dict['md5'])
  File "/home/setepenre/miniconda3/lib/python3.7/site-packages/torchvision/datasets/utils.py", line 248, in download_and_extract_archive
    download_url(url, download_root, filename, md5)
  File "/home/setepenre/miniconda3/lib/python3.7/site-packages/torchvision/datasets/utils.py", line 96, in download_url
    raise e
  File "/home/setepenre/miniconda3/lib/python3.7/site-packages/torchvision/datasets/utils.py", line 84, in download_url
    reporthook=gen_bar_updater()
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 247, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 222, in urlopen
    return opener.open(url, data, timeout)
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 503, in _call_chain
    result = func(*args)
  File "/home/setepenre/miniconda3/lib/python3.7/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found
0it [00:01, ?it/s]

Duplicate of https://github.com/pytorch/vision/issues/1453


Remove unused imports in cond_with_error

Since nom 4.0, using cond_with_error produces warnings of unused imports:
warning: unused imports: `Convert`, `Err`, `IResult`, `Needed`
...
4   |                     | use $ crate :: lib :: std :: result :: Result :: * ; use $ crate :: {
5   |                     | Convert , Err , Needed , IResult } ; if $ cond {
    |                     | ^^^^^^^   ^^^   ^^^^^^   ^^^^^^^
in this expansion of `cond_with_error!` (#6)

    = note: #[warn(unused_imports)] on by default
    = note: this error originates in a macro outside of the current crate (in Nightly builds, run with -Z external-macro-backtrace for more info)

(output stripped for readability)
This (trivial) PR removes the unused imports.

thanks!


Add iree-reduce tool

This patch adds a "iree-reduce" tool, which reduces a given test case to another test case, which is smaller in size, but still gives the same error.
A sample reducing can be found in samples/iree-reduce, which is a testing bug I actually had while helping with an integrate, where the lowering of the arith.max for CUDA was not implemented in LLVM for older architectures.
The implementation is based on llvm-reduce's implementation of Delta Debugging Algorithm (https://en.wikipedia.org/wiki/Delta_debugging).

Currently, I'm looking for high level comments on the reducing infrastructure. Known things:

The build system is not very well written and piggybacks on the compiler driver. This should be fixed before landing.
The code needs to be more commented along with a "tutorial" on how to write a Delta strategy.

Other comments are very welcome on how we should build this infrastructure.

Reasons why I decided not to build on mlir-reduce:

mlir-reduce uses a reduction pattern interface. Reduction methods can be much more than just rewrite patterns and I find writing Delta passes much easier and more intuitive.
mlir-reduce is not the most flexible tool to do things. It also uses the Pass infrastructure, and I wanted more control than that.
Easier to iterate on IREE tool to show what works for us and later upstream it.

Could you perhaps in the description say what kind of reductions it already supports? Perhaps a demo snippet showing what it could reduce and what not. Potentially more appropriate for a README here too.
(AFK but was going to check if this is on bytecode now, if the tool to execute could be fed in, e.g., that would have reduced your needing to rebase often section but could come at some other cost)

I addressed all comments, as well as updated the patch description to better reflect today's state. Thank you for all the feedback on the patch!
The patch should be ready for another review pass on it.

Looks like Ben's comments were addressed...

Nice! Thanks for addressing all of the comments :D


Lack of handles on order item option model

It's good to have a handle on product option, but not having it on order item option means that you can only use the option name to process the options on the submitted orders, which defeats the purpose of having the handles. This PR adds a handle to order item option model and sets it to the handle of the product option.

https://github.com/concrete5-community-store/community_store/pull/570


[SPARK-18670][SS]Limit the number of StreamingQueryListener.StreamProgressEvent when there is no data

What changes were proposed in this pull request?
This PR adds a sql conf spark.sql.streaming.noDataReportInterval to control how long to wait before outputing the next StreamProgressEvent when there is no data.
How was this patch tested?
The added unit test.

cc @tdas

Test build #69505 has started for PR 16108 at commit 4ace2bd.

Test build #69512 has started for PR 16108 at commit 3425231.

Test build #69505 has finished for PR 16108 at commit 4ace2bd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69505/
Test PASSed.

Test build #69512 has finished for PR 16108 at commit 3425231.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69512/
Test PASSed.

Test build #69526 has started for PR 16108 at commit f7b8755.

Test build #69526 has finished for PR 16108 at commit f7b8755.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69526/
Test PASSed.

Test build #69545 has started for PR 16108 at commit be3737f.

Test build #69545 has finished for PR 16108 at commit be3737f.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/69545/
Test PASSed.

LGTM. Merging to master and 2.1


[1.2.x] Bump Mima check to 1.2.6

Contributor Checklist

[NA] Did you add Scaladoc to every public function/method?
[NA] Did you update the FIRRTL spec to include every new feature/behavior?
[NA] Did you add at least one test demonstrating the PR?
[NA] Did you delete any extraneous printlns/debugging code?
[x] Did you specify the type of improvement?
[x] Did you state the API impact?
[x] Did you specify the code generation impact?
[x] Did you request a desired merge strategy?
[ ] Did you add text to be included in the Release Notes for this change?

Type of Improvement

test update

API Impact
None

Backend Code Generation Impact
None
Desired Merge Strategy

Squash: The PR will be squashed and merged (choose this if you have no preference.

Release Notes

Reviewer Checklist (only modified by reviewer)

[ ] Did you add the appropriate labels?
[ ] Did you mark the proper milestone (1.2.x, 1.3.0, 1.4.0) ?
[ ] Did you review?
[ ] Did you check whether all relevant Contributor checkboxes have been checked?
[ ] Did you mark as Please Merge?

Closed and reopened to trigger CI now that it's working


Desktop UX Improvements: Managing servers

This ticket is one of three improvement tickets planned for Desktop user experience.

[x] Navigation bar -- PR #415
[x] Adding a new server from the “+” icon -- PR #415
[x] Opening an app with no servers in Settings page -- PR #435

PART 1 - Navigation bar:
The navigation bar will display on top of the app itself similar to browser tabs and will have the server display name as specified on the Settings page.
This bar will be hidden if there is just one server specified on the Settings page.
The UI screen below shows:

two servers, the first and active server is “Pre-release” and the second is “Bladekick”
a “+” button beside the second tab which can be used to add a new server

PART 2 - Adding a new server from the “+” icon
If a user clicks on the “+” icon, they can enter the name and URL similar to the Settings page. The dialog should be the same from #400, PART 3

After clicking “Add”, the new server will appear on the right-most tab of the navigation bar, and the sign in page will be displayed to the user.
PART 3 - Opening an app with no servers in Settings page
If a user has no servers on the Settings page, a dialog will automatically appear asking them to add a server. The dialog is the same from PART 2 in this ticket (and from #400, PART 3)
Once done, the server is added to the Server Management section.
Note: This also covers the case when a user starts the app for the first time.

Note that "team management" in the desktop app will change to "server management" once team sidebar PR on the platform repo is merged, which adds an additional layer on the left hand sidebar for team management

Future improvements:
Allow users to reorder servers directly on the Server Management page and on the navigation bar. https://mattermost.uservoice.com/forums/306457-general/suggestions/15771343-allow-reordering-teams-in-desktop-application

Closed via #415 and #435, thanks all!


Docs structure: re-apply the Divio structure

Is your feature request related to a problem?
The current docs structure is based on the Divio documentation structure, however some areas have started to be mixed up.
Solution you'd like
Re-apply the Divio documentation structure to the docs and have the different topics re-ordered.
Also, whenever possible, add a Next Steps section to docs in order to provide the reader with a "suggested reading path".
Alternatives you've considered
N/A
Anything else?
No response

While the majority of the structure is in place, here is my suggestion for making it more adapted to Divio documentation structure (it will bring a better cohesion of the docs).
Main sections:

Section
Type
Current placement
New placement
Comment

Introduction
main
root
root
no change

Quick Start
main
root
root
no change

Installation
main
root
root
no change

System Requirements
sub
Installation
Reference
The System requirements should be moved as a reference as they might be different depending on the Kubernetes cluster used

Method 1
sub
Installation
Installation
The generic installation method should stay here

Method 2
sub
Installation
Installation
The generic installation method should stay here

Rancher
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

CLI
sub
Installation
Installation
The generic installation method should stay here

Custom DNS
sub
Installation
Installation
The generic installation method should stay here

Magic DNS
sub
Installation
Installation
The generic installation method should stay here

Public Clouds
sub
Installation
HowTos
The current documentation might be split, where the requirements would move to System requirements > Cloud providers, and like the local Kubernetes clusters, each cloud should have its page

RKE2
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

K3d
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

K3s
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

Rancher Desktop
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

Minikube
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

Wordpress
sub
Installation
HowTos
This installation addresses a specific case and should be considered as solving a "problem"/question

Uninstall
main
root
Installation
While this is an important topic, I think it belongs to the Installation section as it's part of the "Epinio app lifecycle"

Explanations
main
root
root
no change

Advanced topics
sub
Explanations
Explanations
no change

Configuration
sub
Explanations
Reference
The configuration describes the generic settings and should be considered as a reference

Detailed Push Process
sub
Explanations
Explanations
no change

Principles
sub
Explanations
Explanations
no change

Security
sub
Explanations
Explanations
no change

Windows
sub
Explanations
HowTos
While it's not explaining an installation method, it still is a specific case

HowTos
main
root
root
no change

overall
sub
HowTos
HowTos
no change

Reference
main
root
root
no change

overall
sub
Reference
Reference
no change

Please note that currently we don't have a Tutorial section, however I thought of Installation as part of it. Still, the installation is a very important part and it makes total sense to keep it at root level.
Any comments are very welcome and please note that I addressed the sections by looking at the current content and thought of the potential modifications it will get.

The "overall" section seems a bit weird to me in the reference category but my understanding of "reference" is that it should be exhaustive and overall is an overview.
But it's just a feeling

@olblak the "overall" on my first post was to mention the current pages 😓 ... it came out weird on the ASCII tree indeed, sorry


Parameter that is both required and read-only

Let's say I have a spec that has a POST /entity endpoint, whose body is #/definitions/Entity. Entity is a schema that's an object with an id key, which is both required and read-only; my intention is that this key is required in responses, but forbidden in requests. This does not seem supported by pyswagger — as long as I have the key in the required list, it will not validate a request that does not have it, even though readOnly: true forbids it in a request.
...
definitions:
  entity:
    type: object
      required:
        id
      properties:
        id:
          type: integer
          readOnly: true
        name:
          type: string
...

Rationale: It is not possible to provide an id when creating an entity, and when updating an entity its id will be provided in the URL and it makes no sense to allow changing the id (i.e. provide one id in the URL and a different id in the body), so there is no legitimate situation where that id will make sense in a request, but it is required to be present in a response body so that unit tests using the API spec will fail if they don't see it there.

Hi, I would sorry to say that both 'required' and 'readOnly' for Schema object is not implemented yet in pyswagger, I would include these fix in next release.
I'm a little bit busy these days so can't dig into this issue more deep, but I think a property which is both 'required' and 'readOnly' is kind of conflict, and it seems that the spec says it's not legal, too

Relevant only for Schema "properties" definitions. Declares the property as "read only". This means that it MAY be sent as part of a response but MUST NOT be sent as part of the request. Properties marked as readOnly being true SHOULD NOT be in the required list of the defined schema. Default value is false.

(refer to https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#fixed-fields-13)
In your use case, I looked for similar case in issues list of swagger spec, and got this https://github.com/OAI/OpenAPI-Specification/issues/228, and per webron's comment, they discussed more fine grained control, and settle down to current version. IMO, just specify 'readOnly' is enough in your case.

fix and test is included in latest build, please feel free to reopen this case if not work


Add the methods "remove" and "contains_key"

Hey! The previous type map implementation used by serenity provided methods to remove a value from the map and check if a key is present in the map, in accordance with the standard library's map interface (remove, contains_key).
This PR aims to re-introduce those methods in the current type map implementation, as they are useful for some situations.
The functionality provided here is not really new - you can implement contains_key in terms of get (get::<T>().is_some()), and you can remove entries through OccupiedEntry::remove, although a bit tedious. Apart from convenience however, the implementation in this PR does allow you to get the original (owned) value back from the map - which is nice for types which are not Copyable, or not even Cloneable.
One could even think about changing the interface of OccupiedEntry::remove to also return the original value - just like the original from HashMap - but since that changes the existing API, I considered it out of scope for this PR and something that could be done separately.

Thanks.

Thanks acdenis :)


Remove unnecessary CMake version checks

Description
Now that the project requires CMake version 3.15, all of these checks are guaranteed to eveluate one way or the other so they can be removed.
Tasks

[x] Tested on Linux
[x] Tested on Windows
[x] Tested on macOS
[x] Tested on iOS
[x] Tested on Android

SFMLBuildMaster: Build this please.

Thanks for the clean up 🧹


服务器正在启动中...

问题描述
服务器正在启动中...
问题环境

版本号：talebook/talebook                  latest    fb0fa9c485c3   3 weeks ago     2.18GB

URL：出现异常时的URL

运行系统：NAS上的DOCKER

访问设备：Windows10 x64

浏览器版本：版本 5.1.1130.128（正式版本） （64 位） (Chromium 118.0.5993.159)

书籍信息：如果是特定的书异常，请提供「书名」和「作者名」

复现步骤

先做啥
再做啥
看到报错XXX

docker run -d --name talebook -p 8080:80 -v /volume1/Obsidian:/data talebook/talebook
截图
请上传报错的截图

报错信息
root@DSM7:~# docker logs d4dd3464113f
updating '/data/' permission to 1000:1000
ERROR:root:Traceback (most recent call last):
File "/var/www/talebook/webserver/loader.py", line 40, in loadfile
import auto
ModuleNotFoundError: No module named 'auto'
INFO:root:Init library with [/data/books/library/]
INFO:root:Init AuthDB  with [sqlite:////data/books/calibre-webserver.db]
INFO:root:Init Static  with [/var/www/talebook/webserver/resources]
INFO:root:Init HTML    with [/var/www/talebook/webserver/../app/dist]
INFO:root:Init Nuxtjs  with [/var/www/talebook/webserver/../app/.env]
INFO:root:Bind modles._session()
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 146, in init
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3300, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 180, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 674, in init
self.__connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 901, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 896, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 643, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/default.py", line 620, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/www/talebook/server.py", line 13, in
sys.exit(webserver.main.main())
File "/var/www/talebook/webserver/main.py", line 219, in main
app = make_app()
File "/var/www/talebook/webserver/main.py", line 149, in make_app
models.user_syncdb(engine)
File "/var/www/talebook/webserver/models.py", line 315, in user_syncdb
Base.metadata.create_all(engine)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 5857, in create_all
bind._run_ddl_visitor(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3250, in _run_ddl_visitor
with self.begin() as conn:
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3240, in begin
with self.connect() as conn:
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3276, in connect
return self._connection_cls(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 148, in init
Connection._handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 2440, in _handle_dbapi_exception_noconnection
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 146, in init
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3300, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 180, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 674, in init
self.__connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 901, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 896, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 643, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/default.py", line 620, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: https://sqlalche.me/e/20/e3q8)
ERROR:root:Traceback (most recent call last):
File "/var/www/talebook/webserver/loader.py", line 40, in loadfile
import auto
ModuleNotFoundError: No module named 'auto'
INFO:root:Init library with [/data/books/library/]
INFO:root:Init AuthDB  with [sqlite:////data/books/calibre-webserver.db]
INFO:root:Init Static  with [/var/www/talebook/webserver/resources]
INFO:root:Init HTML    with [/var/www/talebook/webserver/../app/dist]
INFO:root:Init Nuxtjs  with [/var/www/talebook/webserver/../app/.env]
INFO:root:updating configs ...
ERROR:root:service register <auto_fill_all>
ERROR:root:service register <auto_fill>
ERROR:root:service register <send_mail>
ERROR:root:service register <send_book>
INFO:root:done
Restarting nginx: nginx.
2024-06-26 12:24:59,903 INFO Included extra file "/etc/supervisor/conf.d/talebook.conf" during parsing
2024-06-26 12:24:59,903 INFO Set uid to user 0 succeeded
2024-06-26 12:24:59,933 INFO RPC interface 'supervisor' initialized
2024-06-26 12:24:59,933 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-06-26 12:24:59,934 INFO supervisord started with pid 1
2024-06-26 12:25:00,937 INFO spawned: 'tornado' with pid 61
2024-06-26 12:25:02,295 INFO success: tornado entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2024-06-26 12:25:05,403 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:25:06,404 INFO spawned: 'tornado' with pid 68
2024-06-26 12:25:07,157 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:25:08,160 INFO spawned: 'tornado' with pid 75
2024-06-26 12:25:09,144 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:25:11,148 INFO spawned: 'tornado' with pid 82
2024-06-26 12:25:11,868 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:25:14,872 INFO spawned: 'tornado' with pid 89
2024-06-26 12:25:15,549 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:25:16,551 INFO gave up: tornado entered FATAL state, too many start retries too quickly
2024-06-26 12:27:21,670 WARN received SIGTERM indicating exit request
usermod: no changes
ERROR:root:Traceback (most recent call last):
File "/var/www/talebook/webserver/loader.py", line 40, in loadfile
import auto
ModuleNotFoundError: No module named 'auto'
INFO:root:Init library with [/data/books/library/]
INFO:root:Init AuthDB  with [sqlite:////data/books/calibre-webserver.db]
INFO:root:Init Static  with [/var/www/talebook/webserver/resources]
INFO:root:Init HTML    with [/var/www/talebook/webserver/../app/dist]
INFO:root:Init Nuxtjs  with [/var/www/talebook/webserver/../app/.env]
INFO:root:Bind modles._session()
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 146, in init
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3300, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 180, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 674, in init
self.__connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 901, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 896, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 643, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/default.py", line 620, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
sqlite3.OperationalError: unable to open database file
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/var/www/talebook/server.py", line 13, in
sys.exit(webserver.main.main())
File "/var/www/talebook/webserver/main.py", line 219, in main
app = make_app()
File "/var/www/talebook/webserver/main.py", line 149, in make_app
models.user_syncdb(engine)
File "/var/www/talebook/webserver/models.py", line 315, in user_syncdb
Base.metadata.create_all(engine)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/sql/schema.py", line 5857, in create_all
bind._run_ddl_visitor(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3250, in _run_ddl_visitor
with self.begin() as conn:
File "/usr/lib/python3.9/contextlib.py", line 117, in enter
return next(self.gen)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3240, in begin
with self.connect() as conn:
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3276, in connect
return self._connection_cls(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 148, in init
Connection._handle_dbapi_exception_noconnection(
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 2440, in _handle_dbapi_exception_noconnection
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 146, in init
self._dbapi_connection = engine.raw_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/base.py", line 3300, in raw_connection
return self.pool.connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 449, in connect
return _ConnectionFairy._checkout(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 712, in checkout
rec = pool._do_get()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 180, in _do_get
self._dec_overflow()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/impl.py", line 177, in _do_get
return self._create_connection()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 674, in init
self.__connect()
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 901, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/util/langhelpers.py", line 146, in exit
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/pool/base.py", line 896, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/create.py", line 643, in connect
return dialect.connect(*cargs, **cparams)
File "/usr/local/lib/python3.9/dist-packages/sqlalchemy/engine/default.py", line 620, in connect
return self.loaded_dbapi.connect(*cargs, **cparams)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file
(Background on this error at: https://sqlalche.me/e/20/e3q8)
ERROR:root:Traceback (most recent call last):
File "/var/www/talebook/webserver/loader.py", line 40, in loadfile
import auto
ModuleNotFoundError: No module named 'auto'
INFO:root:Init library with [/data/books/library/]
INFO:root:Init AuthDB  with [sqlite:////data/books/calibre-webserver.db]
INFO:root:Init Static  with [/var/www/talebook/webserver/resources]
INFO:root:Init HTML    with [/var/www/talebook/webserver/../app/dist]
INFO:root:Init Nuxtjs  with [/var/www/talebook/webserver/../app/.env]
INFO:root:updating configs ...
ERROR:root:service register <auto_fill_all>
ERROR:root:service register <auto_fill>
ERROR:root:service register <send_mail>
ERROR:root:service register <send_book>
INFO:root:done
Restarting nginx: nginx.
2024-06-26 12:27:36,049 INFO Included extra file "/etc/supervisor/conf.d/talebook.conf" during parsing
2024-06-26 12:27:36,049 INFO Set uid to user 0 succeeded
2024-06-26 12:27:36,060 INFO RPC interface 'supervisor' initialized
2024-06-26 12:27:36,060 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2024-06-26 12:27:36,061 INFO supervisord started with pid 1
2024-06-26 12:27:37,064 INFO spawned: 'tornado' with pid 52
2024-06-26 12:27:37,982 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:27:38,985 INFO spawned: 'tornado' with pid 60
2024-06-26 12:27:39,678 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:27:41,683 INFO spawned: 'tornado' with pid 67
2024-06-26 12:27:42,331 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:27:45,337 INFO spawned: 'tornado' with pid 74
2024-06-26 12:27:46,045 INFO exited: tornado (exit status 1; not expected)
2024-06-26 12:27:47,046 INFO gave up: tornado entered FATAL state, too many start retries too quickly
root@DSM7:~#
日志
请上传 talebook.log 文件，或截取日志中 提示 Exceptions 的部分
书籍文件
如果是特定某本书出现异常，请上传该书籍文件
其他
如果是比较复杂的问题，可以把整个书库 /books/library/metadata.db 数据库文件上传，以便作者复现异常场景

你这个问题，看起来是挂载的目录中残留了之前的数据文件，但是又有缺失 /volume1/Obsidian/books/settings/auto.py

我也是这个问题感觉是bug，因为我已经尝试删除挂载目录甚至清理了docker的目录

经过我不挂载目录发现正常，因此推测是目录权限问题，我是群晖，因此将挂载目录设置为everyone即可了，成功解决问题

我也是这个问题

经过我不挂载目录发现正常，因此推测是目录权限问题，我是群晖，因此将挂载目录设置为everyone即可了，成功解决问题

挂载权限问题。


Binary/Cereal

Why do you use binary rather than cereal, given that you are now doing an extra conversion between strict and lazy ByteStrings?

Good point. binary isn't used internally, there are only some helper functions which use it. They are helpful when your whole application uses binary for serialization (which was true for us earlier).
I think those helpers should be removed and dependency on binary should be dropped.

Btw, in our application we don't use binary anymore, we are using cborg.


Range and span expressions

Manipulation of span in parser is something utterly important. We feel the need to propose some support to handle span more easily. We therefore propose two expression extensions:
(... e1 e2)

which have the type (Range<S>, T1, T2) where Range<S> is as defined in #85 and
(.. e1 e2)

with the type (Span, T1, T2) where Span is obtained by calling range.span().
Design

Add an expression StreamRange(usize) in the AST.
Add a type Span along with Range.
Force .. and ... to start a sequence, e1 .. e2 is disallowed because it leads to visual ambiguities.
We do not propose Range<(T1, T2)> (for example) because we feel it's inconvenient, it forces the user to extract the span and the value in the semantic action.

It was first proposed in #13 to infer span, we propose here a better control while the span type is still correctly inferred and propagated.

Triage of issues: I abandon the syntax ... (only .. remains) for now. It is subsumed by a more general mechanism which allows us to call external parsers inside a Oak rule.

Triage of issues: I abandon the syntax ... (only .. remains) for now. It is subsumed by a more general mechanism which allows us to call external parsers inside a Oak rule.


Track app status

It appears bad when apps supported as part of the system do not work, or even open.  Two recent examples are the opendap app that acts on multidimensional resources/files, and Jupyterhub.   I suggest:

An "app heartbeat" function that periodically (e.g. daily) automatically attempts to open each app to verify that it is still up and if not reports this to administrators.
Use tracking functionality to monitor and report which apps are up and down and for how long.
Setting the permissions for all approved app instances to have an appropriate system administrator an owner, so that when an approved app is offline the permissions can be set to private to limit attempts to use that result in an error, and that if warranted, a message of the day message posted.
An alternative to message of the day, is an "offline" setting in the app instance that rather than attempting to open an app presents an "offline" message.

Would an external service eg http://pingdom.com suffice here? What monitoring service is being used for HS proper?

@alexlemann - HydroShare is using django-heartbeat
We have a Jenkins monitoring job that runs at a 5 minute interval to check the status. On failure it attempts a restart HydroShare and emails a subset of the HydroShare team regarding status.

@Castronova giving this to you.  I'd like this to be part of CUAHSI system tracking functionality.  If too much to do for 1.11, you could put later.

Could something like Datadog, New Relic or other DevOps tool serve for both internal engineering and also provide a lighter public portal?

@Castronova Can we close this.  It is not repository role to track apps.

closing this, it is stale we can reopen if needed


bst sometimes modifies source track parameters

See original issue on GitLab
In GitLab by [Gitlab user @tlater] on Nov 30, 2017, 12:18
I ran into this attempting to build the baserock project - it seems that sometimes bst will modify the track parameter of a source.
Using bst track busybox.bst on the following mini-project:
project.conf:
name: baserock

aliases:
  upstream: git://git.baserock.org/delta/

# All the elements go here
element-path: ./elements

elements/busybox.bst:
kind: manual

sources:
- kind: git
  url: upstream:busybox
  track: 1_23_1
  ref: 1ecfe811fe2f70380170ef7d820e8150054e88ca

Results in the following changes:
6,7c6,7
<   track: 1_23_1
<   ref: 1ecfe811fe2f70380170ef7d820e8150054e88ca
---
>   track: 1_2_3_1
>   ref: 1231a93bd05f8999fb25560fae17e7bc1d272a4f

The ref update is expected, but the given branch does not exist, and actually running a build results in a failure since the commit does not exist.
Presumably this happens due to ruamel.yaml attempting to escape something when writing back the element after acquiring a new ref. The bug is probably in that project, but we should look a bit deeper into it before reporting :)

In GitLab by [Gitlab user @tristanvb] on Jan 15, 2018, 07:22
Indeed, verified, this happens without any BuildStream in play. Note that not only is it serialized without retaining the original value, the incorrect value is also initially loaded (as demonstrated in my test case below).
Marking this as critical too, because it inflicts damage/loss to the user's data.
I've filed an upstream bug report
Here is a simple test case to reproduce the issue with ruamel.yaml:
#!/usr/bin/env python3

import io
import sys
from ruamel import yaml

raw_data = "the_string: 1_23_4"

print("Raw input:")
print("----------")
print(raw_data)

with io.StringIO(raw_data) as f:
    data = yaml.round_trip_load(f)

print("")
print("Incorrectly loaded string value:")
print("--------------------------------")
print(data['the_string'])

print("")
print("The dump result is also not the same as the input:")
print("--------------------------------------------------")
yaml.round_trip_dump(data, sys.stdout)

In GitLab by [Gitlab user @tristanvb] on Jan 15, 2018, 09:18
Additional note: This is partly because in the YAML 1.1 specification, underscores are intended to be ignored from numeric expressions.
This doesnt change much, but it may help to explain why this is happening (it's still a bug that serializing in round trip mode loses the original formatting).
Probably a good fix on our side for this is to find a way to inform ruamel.yaml that certain specific fields are intended to be read as strings; ideally transparently through our internal _yaml wrapper API, which already expresses the desired type of values loaded.

In GitLab by [Gitlab user @knownexus] on Jan 19, 2018, 09:03
mentioned in merge request !245

In GitLab by [Gitlab user @knownexus] on Feb 12, 2018, 17:25
Have we considered the option of simply having the tracking number in quotes?
If we are always going to read it as a string, this could be a neat fix without relying on yaml changes

In GitLab by [Gitlab user @juergbi] on Feb 13, 2018, 11:02

Have we considered the option of simply having the tracking number in quotes?

As the format is already considered stable we cannot suddenly start requiring quotes. And if we keep allowing unquoted values and don't fix it in the code, people will continue to run into this issue.

In GitLab by [Gitlab user @richardmaw-codethink] on Feb 14, 2018, 16:13
If we can hobble the ruamel resolver or constructor ([Gitlab user @knownexus] seems to be having some success with the constructor) to load numbers as strings then we should be able to pass preserve_quotes=True to ruamel.yaml.load.
I don't know whether it will prevent it from adding quotes to numbers it has loaded as a string, but it should prevent it from converting double quotes to single ones and stripping quotes from strings that are redundantly quoted.

In GitLab by [Gitlab user @richardmaw-codethink] on Feb 14, 2018, 18:50
If we disable number parsing more completely like in https://gitlab.com/snippets/1698876 then numbers round-trip without change.

In GitLab by [Gitlab user @knownexus] on Feb 15, 2018, 13:42
assigned to [Gitlab user @knownexus]

In GitLab by [Gitlab user @juergbi] on Feb 15, 2018, 17:14
closed via commit 57c8b8884828b900d64188a3032f5adb04d46372

In GitLab by [Gitlab user @samthursfield] on Feb 15, 2018, 18:18
mentioned in commit 57c8b8884828b900d64188a3032f5adb04d46372

In GitLab by [Gitlab user @tristanvb] on Feb 16, 2018, 12:39
Reopening this because it's a severe bug, and as such at least this should be ported back to bst-1.0 branch before considering this to be closed.

In GitLab by [Gitlab user @tristanvb] on Feb 16, 2018, 12:39
reopened

In GitLab by [Gitlab user @knownexus] on Mar 21, 2018, 11:51
Is this still an issue, or can it now be closed?

In GitLab by [Gitlab user @toscalix] on May 23, 2018, 17:35
assigned to [Gitlab user @tristanvb]

In GitLab by [Gitlab user @toscalix] on May 23, 2018, 17:35
unassigned [Gitlab user @tristanvb]

In GitLab by [Gitlab user @toscalix] on May 23, 2018, 17:35
assigned to [Gitlab user @tristanvb] and unassigned [Gitlab user @knownexus]

In GitLab by [Gitlab user @LaurenceUrhegyi] on Aug 9, 2018, 18:04
assigned to [Gitlab user @knownexus]

In GitLab by [Gitlab user @LaurenceUrhegyi] on Aug 9, 2018, 18:05
unassigned [Gitlab user @tristanvb]

In GitLab by [Gitlab user @LaurenceUrhegyi] on Aug 9, 2018, 18:05
Was this back-ported? If not then can we do so and close the ticket please?

In GitLab by [Gitlab user @tristanvb] on Aug 9, 2018, 21:09
[Gitlab user @LaurenceUrhegyi], no need to backport anything to bst-1.0 anymore I think, we can put this one to rest and discontinue bst-1.0.

In GitLab by [Gitlab user @tristanvb] on Aug 9, 2018, 21:09
closed


Drab.Config will probably fail with releases

When an Elixir release is built with Exrm or Distillery, Mix is not available. So, the following code will likely fail.
defmodule Drab.Config do
  # ....
  def app_name() do
    Mix.Project.config()[:app] || raise """
      Can't find the main application name. Please check your mix.exs
      """
  end

Fixed in v0.5.1


Fails with latest node version

Running the make process fails with the latest node version. The setup worked with an older node version
Please provide the following info:

MBT Version:  0.2.5
OS Version: macOS 10.14.5
If possible, provide a link to the project. Can use any project or this one https://github.com/IObert/Beershop

Steps to Reproduce:

create project
mbt init
make -f Makefile.mta p=cf

[2019-05-17 18:08:35]  INFO validating the MTA project
[2019-05-17 18:08:35]  INFO validating the MTA project
[2019-05-17 18:08:35]  INFO building the "beershop-db" module...
[2019-05-17 18:08:35]  INFO executing the [npm install] command...
..
><EMAIL_ADDRESS>install /Users/d056949/Desktop/beershop/db/node_modules/@sap/hdi-deploy/node_modules/@sap/hana-client
> node build.js

..Error: Command failed: node-gyp build
In file included from ../src/hana.cpp:5:
../src/h/hana_utils.h:298:11: error: no template named 'Handle'
                        Handle<Value>                           arg,
                        ^
../src/h/hana_utils.h:301:6: error: redefinition of 'getBindParameters' as different kind of symbol
bool getBindParameters( Handle<Value>                           arg,
     ^
../src/h/hana_utils.h:294:6: note: previous definition is here
bool getBindParameters( std::vector<char*>                      &string_params,
     ^
../src/h/hana_utils.h:301:25: error: use of undeclared identifier 'Handle'
bool getBindParameters( Handle<Value>                           arg,
                        ^
../src/h/hana_utils.h:301:32: error: 'Value' does not refer to a value
bool getBindParameters( Handle<Value>                           arg,
                               ^
/Users/d056949/.node-gyp/12.2.0/include/node/v8.h:2240:17: note: declared here
class V8_EXPORT Value : public Data {
                ^
In file included from ../src/hana.cpp:5:
../src/h/hana_utils.h:301:43: error: use of undeclared identifier 'arg'
bool getBindParameters( Handle<Value>                           arg,
                                                                ^
../src/h/hana_utils.h:302:65: error: expected '(' for function-style cast or type construction
                        int                                     row_param_count,
                        ~~~                                     ^
../src/h/hana_utils.h:303:65: error: expected '(' for function-style cast or type construction
                        std::vector<dbcapi_bind_data*>          &params,
                        ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~          ^
../src/h/hana_utils.h:303:66: error: use of undeclared identifier 'params'
                        std::vector<dbcapi_bind_data*>          &params,
                                                                 ^
../src/h/hana_utils.h:304:45: error: expected '(' for function-style cast or type construction
                        std::vector<size_t> &                   buffer_size );
                        ~~~~~~~~~~~~~~~~~~~ ^
../src/h/hana_utils.h:304:63: error: use of undeclared identifier 'buffer_size'
                        std::vector<size_t> &                   buffer_size );
                                                                ^
../src/h/hana_utils.h:310:26: error: use of undeclared identifier 'Handle'
bool getInputParameters( Handle<Value>                      arg,
                         ^
../src/h/hana_utils.h:310:33: error: 'Value' does not refer to a value
bool getInputParameters( Handle<Value>                      arg,
                                ^
/Users/d056949/.node-gyp/12.2.0/include/node/v8.h:2240:17: note: declared here
class V8_EXPORT Value : public Data {
                ^
In file included from ../src/hana.cpp:5:
../src/h/hana_utils.h:310:61: error: use of undeclared identifier 'arg'
bool getInputParameters( Handle<Value>                      arg,
                                                            ^
../src/h/hana_utils.h:311:57: error: expected '(' for function-style cast or type construction
                         std::vector<dbcapi_bind_data*> &   params,
                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
../src/h/hana_utils.h:311:61: error: use of undeclared identifier 'params'
                         std::vector<dbcapi_bind_data*> &   params,
                                                            ^
../src/h/hana_utils.h:312:26: error: unexpected type name 'dbcapi_stmt': expected expression
                         dbcapi_stmt *                      stmt,
                         ^
../src/h/hana_utils.h:312:61: error: use of undeclared identifier 'stmt'; did you mean 'stat'?
                         dbcapi_stmt *                      stmt,
                                                            ^~~~
                                                            stat
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/stat.h:361:9: note: 'stat' declared here
int     stat(const char *, struct stat *) __DARWIN_INODE64(stat);
        ^
In file included from ../src/hana.cpp:5:
../src/h/hana_utils.h:313:30: error: expected '(' for function-style cast or type construction
                         int &                              errCode,
                         ~~~ ^
../src/h/hana_utils.h:313:61: error: use of undeclared identifier 'errCode'; did you mean 'strmode'?
                         int &                              errCode,
                                                            ^~~~~~~
                                                            strmode
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/string.h:164:7: note: 'strmode' declared here
void     strmode(int __mode, char *__bp);
         ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[1]: *** [Release/obj.target/hana-client/src/hana.o] Error 1
gyp ERR! build error
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/d056949/.nvm/versions/node/v12.2.0/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:262:23)
gyp ERR! stack     at ChildProcess.emit (events.js:196:13)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:257:12)
gyp ERR! System Darwin 18.6.0
gyp ERR! command "/Users/d056949/.nvm/versions/node/v12.2.0/bin/node" "/Users/d056949/.nvm/versions/node/v12.2.0/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "build"
gyp ERR! cwd /Users/d056949/Desktop/beershop/db/node_modules/@sap/hdi-deploy/node_modules/@sap/hana-client
gyp ERR! node -v v12.2.0
gyp ERR! node-gyp -v v3.8.0
gyp ERR! not ok

    at ChildProcess.exithandler (child_process.js:304:12)
    at ChildProcess.emit (events.js:196:13)
    at maybeClose (internal/child_process.js:1011:16)
    at Socket.<anonymous> (internal/child_process.js:416:11)
    at Socket.emit (events.js:196:13)
    at Pipe.<anonymous> (net.js:586:12) {
  killed: false,
  code: 1,
  signal: null,
  cmd: 'node-gyp build'
}   CXX(target) Release/obj.target/hana-client/src/hana.o

npm WARN deploy@ No description
npm WARN deploy@ No repository field.
npm WARN deploy@ No license field.

npm ERR! code ELIFECYCLE
npm ERR! errno 255
npm ERR<EMAIL_ADDRESS>install: `node build.js`
npm ERR! Exit status 255
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR!      /Users/d056949/.npm/_logs/2019-05-17T16_08_39_826Z-debug.log
[2019-05-17 18:08:39] ERROR build of the "beershop-db" module failed when executing commands: execution of the "/Users/d056949/.nvm/versions/node/v12.2.0/bin/npm" command failed while waiting for finish: exit status 255
make: *** [beershop-db] Error 1

HI,
From the error, it seems that the problem is due to API incompatibility (Hana-client) which probably changed during major released version ( node v12.x.x), you can try it using the following:

cd db
run npm install

You should get the same error as the mbt is utilizing the installed node on your mac.
If you have any doubts, please let me know.
Regards,
Shimon

You're right. The issue is not related to this project.
Thanks for pointing me in the right direction!

Thanks a lot for this incident. It helped me to get a successful installation of
npm i -g @sap/hdi-deploy

after downgrading from node/12.2.0 to node/10.16.0.


Don't rename crates

Summary of changes
Changes introduced in this pull request:

Fix crates that have been renamed (i.e. crypto -> forest_crypto)

Reference issue to close (if applicable)

Closes
#1706
Other information and links

To verify this PR fixes all renaming issues, use `git grep "package = "

Make sure that your PRs have a line saying "Closes #issue_number". If you put the issue on a separate line, GitHub doesn't understand that they are connected.

In general, do we know the reasoning behind renaming those crates in the first place?

In general, do we know the reasoning behind renaming those crates in the first place?

I assume the initial authors felt it was prettier.


[FEATURE] Support for internal Bitbucket Server?

Is your feature request related to a problem?
Not a technical issue, just a nice-to-have feature.
Describe the solution you'd like
The ability to provide the URL of a private Bitbucket Server instance (probably on the "getting started" window), in order to set up the required Application Permissions on my account.
Describe alternatives you've considered
I'm currently syncing my settings to a repo on my personal GitHub account, and I can provide that URL to my coworkers as necessary.
Additional context
I work on a team that uses an internal Bitbucket Server. In addition to syncing my own personal settings, I intend to create a minimal "beginner" profile that I can share with other members of my team, most of whom are new to VS Code. I can't guarantee that these individuals already have accounts on the public GitHub/GitLab/Bitbucket sites, but I know for certain that they have credentials for our internal server. This feature would mean one less step for them to follow.
Thanks very much for your hard work so far!

@dxman I have added this in v3.9.0.
How to Use

Run Syncify: Download
Click the Download without Login button
Enter the URL of the "beginner" repository

This will download the beginner template, then remove the URL from the settings to prevent accidental uploads to the repo.

If the repository on the BitBucket server is only readable when authenticated, they can use SSH or a personal access token.
SSH Example: git@your.endpoint.com:user/repo.git
PAT Example<EMAIL_ADDRESS>


Refactor UI state management

There isn't much structure to the current way state management is done in the UI. Many callbacks have to be passed around to update parent components, etc.
A principled approach using the appropriate tools (e.g. Redux) would make the app more stable and easier to update.

Some notes on Redux: Redux state store is global and it hits all the loopholes of using global state/variables.
In a prior project, migrating one Redux application into another Redux application was quite messy. In other words, the globalness breaks the principle of componentization. If users want to embed this app into their parent website organically, it might be hard.
Have we looked into React Context ? Did we see any issues Context does not address?

@chance-an I was just throwing Redux out there as an example, I actually don't know enough about it to have an opinion but I do trust yours.
The current way we keep state is also quite messy, and not principled. We do use React Contexts for a number of things, e.g.  logged in user, snack notifications, env variables:
https://github.com/sematic-ai/sematic/blob/97618ab47ea5e8150a01bbf812fe24327ac54678/sematic/ui/src/index.tsx#L109-L111
I never thought of using it more broadly for state management, but that could make sense. It does make everything easier.


Pass default values for Page, Collection

There is only a few default template paths to be expected (explicit is better than...)
Some themes probably have a desired usage and forcing the user to memorize each one (especially when there are multiple themes being used).
Suggestion
ThemeManager could have attributes for
- default_page_template,
- default_collection_template,
- default_archive_template
Each of these would be str|None and only apply if the attribute is supplied

More info:
so if the list of templates are
{
"test.html": "Test1",
"test.html": "Test2",
"test.html": "Test3",
}
The result when calling test.html will be "Test1"
If someone is making a template they have two choices. override or  workaround
Then the user has the same two choices
the order of preference is User --> Templates --> Render Engine
This change would have some values defined by default which would reduce the need to manually define your template all the time
It would also make it so that if you use that object, you can rely on a default template path. If the system defines a default the user and theme maker both know what the default is and can always default their path to it.

This is connected to #228

Okay, so we would be adding those attributes to the ThemeManager class like so:
class ThemeManager:

    engine: Environment
    output_path: str = "output"
    static_paths: set[str|pathlib.Path] = {"static"}
    default_page_template: str | None = None
    default_collection_template: str | None = None
    default_archive_template:  str | None = None

The end goal would be that if a template path is provided for, say, default_page_template--this would mean that the a user would not need to pass in a template value within the Page class? However, if the user does pass a template in through the Page class, it would take precedence over the default?
Would we then control the loaders from the register_themes method in ThemeManager?

Yes one of the things that running register_themes does is add the themes loader to the Environment. It also installs all the filters from the theme.

@tataraba -
I'm a little torn on this as I'm trying to figure out if it's better to ask the user to specify.
Assumption:

Basic page's default template should be page.html which is a built-in template.

WHY THIS ASSUMPTION IS BAD:

Some pages don't need templates and are just generated content.

RSSFeeds are page objects if there was a default template. This could cause issues.

If you wanted to generate a report using render_engine (not sure why you would yet but you could), You may or may not want a template.

If there is a plugin for htmx and you're rendering output from some subpage, you likely don't want that page to be templated

Also currently with render_content the check is (in order):
If a template exists you render from the template and pass in your globals, kwargs, etc.
if no template is given then you render the content (arguably you should also be able to check for data and render it).
Finally if none of those things exist... Raise ValueError.
I think it may be better just to instruct users to


Turn Resuming edit session... into a hyperlink which points to the output channel

Testing #158798

It would be nice if that text is a hyperlink which opens the output channel, similar to Starting Dev container above:

/verified


[WIP] RouteResolver should be less opinionated

Coverage decreased (-1.9%) to 98.077% when pulling e211c9d19bdf06d3d308d79978a7062536b09f65 on renatomefi:feature/less-opinioated-route-resolver into 2e0bbe60cf71f72f50b7d1f637a3ea83bc364de8 on overblog:master.

Coverage decreased (-1.9%) to 98.077% when pulling e211c9d19bdf06d3d308d79978a7062536b09f65 on renatomefi:feature/less-opinioated-route-resolver into 2e0bbe60cf71f72f50b7d1f637a3ea83bc364de8 on overblog:master.

@ooflorent I understand your feeling about the 'default' part, the idea is the RouteResolver to be not influenced by how OverblogGraphQLBundle works, and 'default' is sort of this, so the idea is to make the RouteResolver usable for anything, even the user configuring manual routes inside of it, that's the objective.
Having it in mind do you have a suggestion on how to achieve it?

I'm ok with default. And I'm also ok with null. I'm just saying we should only one option. Since both behave the same, I don't really care of the chosen one.


nextcloud-notify-push: init at 0.5.0

Description of changes
Probably only makes sense with a module
Do the maintainers have any preference of how to go about that?
Separate module? Put it into the nextcloud module? Stuff like that…
Edit: only took me >1.5y since https://github.com/NixOS/nixpkgs/pull/113958#pullrequestreview-597991184 to get this started.
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 201780 run on x86_64-linux 1

1 package built:

nextcloud-notify-push

Is there anything open for this package?  The discussion was about the time someone would add a module integration, which is atm not part of the PR (thus author asked for input). But i dont think that this discussion should hinder the package to get merged, as its already usable if someone does the integration of it manually (something similar to what my module does).

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-already-reviewed/2617/739

I have a module and test nearly done for this. Will push on the weekend or next week.
The package is trivial and IMO doesn't make much sense to have in nixpkgs without a module.

Thanks for the feedback :) i just tried to make sure that its not getting stale, as the package is working. as long as you are still working on it/motivated its super fine for me.
Just let me know if there is something for testing or any other help, i have 2 instances which can serve for tests.

Don't worry. Also, if we consider this ready, I can just merge it ^^

@ajs124 I think after the two suggestions from @Shawn8901 we can merge this. Do you want to do that or should I push them onto the PR?

Not sure if this was already mentioned but should we set services.nextcloud.caching.redis to true?

Not sure if this was already mentioned but should we set services.nextcloud.caching.redis to true?

That alone is sadly not enough beside that config there is additional setup needed. I already tried to point that here https://github.com/NixOS/nixpkgs/pull/201780#discussion_r1050265272
On my opinion it's not quite nicely setup-able by the nextcloud module, so that it can be "just used here". A clean solution (on my opinion) would be to have a "enableRedis" option on the nextcloud module level, so that we can just force enable it when notify-push is enabled.

Ah, forgot that already. I think I am going to add a setting for that after this PR got merged.

@ofborg eval

What we could do, since the whole dependency thing is kind of fragile, even if implemented in nixpkgs, is waiting for redis to come up by looping with redis-cli in a preStart. Although that probably isn't significantly less fragile, either.

I am going ahead and merge this now to not delay it any longer. My additional options will be an extra PR likely with an update.