tts: 0.4.2 -> 0.5.0

Motivation for this change
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Executing pipInstallPhase
/build/source/dist /build/source
Processing ./TTS-0.5.0-cp39-cp39-linux_x86_64.whl
Requirement already satisfied: coqpit in /nix/store/93f3zbaqhgd8pw6f30yknv1cqask02yg-python3.9-coqpit-0.0.14/lib/python3.9/site-packages (from TTS==0.5.0) (0.0.14)
Requirement already satisfied: matplotlib in /nix/store/9g0d6j5wqnglva453nmyk1ks9n1hmsbr-python3.9-matplotlib-3.5.1/lib/python3.9/site-packages (from TTS==0.5.0) (3.5.1)
Requirement already satisfied: jieba in /nix/store/k0ybyym8n253dja2anapzzdiv4qla7hm-python3.9-jieba-0.42.1/lib/python3.9/site-packages (from TTS==0.5.0) (0.42.1)
Requirement already satisfied: inflect in /nix/store/y20j3mswiz4cr98qd4s206956amklfga-python3.9-inflect-5.3.0/lib/python3.9/site-packages (from TTS==0.5.0) (5.3.0)
Requirement already satisfied: webrtcvad in /nix/store/g4p4paavi91ydjqmrwiv1ksh87nhkhbw-python3.9-webrtcvad-2.0.10/lib/python3.9/site-packages (from TTS==0.5.0) (2.0.10)
Requirement already satisfied: tqdm in /nix/store/b8maphcanpc77sbb64xdg9b7j2npv9gi-python3.9-tqdm-4.62.3/lib/python3.9/site-packages (from TTS==0.5.0) (4.62.3)
Requirement already satisfied: mecab-python3 in /nix/store/cfjwwn0fhxf1c79l7fjjqjnhs48z3rwf-python3.9-mecab-python3-1.0.4/lib/python3.9/site-packages (from TTS==0.5.0) (1.0.4)
Requirement already satisfied: cython in /nix/store/qvnzadwl84zk9n84pyw46ll9m5nnvbk5-python3.9-Cython-0.29.24/lib/python3.9/site-packages (from TTS==0.5.0) (0.29.24)
Requirement already satisfied: torch in /nix/store/hk1vbrni06niazz3g98zf6vg3731fhls-python3.9-pytorch-1.9.0/lib/python3.9/site-packages (from TTS==0.5.0) (1.9.0)
Requirement already satisfied: pyworld in /nix/store/8ia12xih945g26iqf6dlm6dbbacfbx1w-python3.9-pyworld-0.3.0/lib/python3.9/site-packages (from TTS==0.5.0) (0.3.0)
Requirement already satisfied: gdown in /nix/store/nqwhpjfnaiyhccpa8f8pl4jnmj8xrj00-gdown-4.2.0/lib/python3.9/site-packages (from TTS==0.5.0) (4.2.0)
Requirement already satisfied: soundfile in /nix/store/4pj877v03scgr6b2irg7a2gkgcjcy3hj-python3.9-soundfile-0.10.3.post1/lib/python3.9/site-packages (from TTS==0.5.0) (0.10.3.post1)
Requirement already satisfied: numpy in /nix/store/wch23ab3h4bhf9lpjhhs6mgs9mcmh5wj-python3.9-numpy-1.21.4/lib/python3.9/site-packages (from TTS==0.5.0) (1.21.4)
Requirement already satisfied: pandas in /nix/store/pibnq49h8xd33z54d4j7ycdv4xgabcif-python3.9-pandas-1.3.4/lib/python3.9/site-packages (from TTS==0.5.0) (1.3.4)
Requirement already satisfied: numba in /nix/store/xj9nkqki4xim1fq421ymmpck5q11s9qc-python3.9-numba-0.54.1/lib/python3.9/site-packages (from TTS==0.5.0) (0.54.1)
Requirement already satisfied: tensorboardX in /nix/store/mg73h9bmvsdcjqj29ahfqsw93ysi2rf7-python3.9-tensorboardx-2.4/lib/python3.9/site-packages (from TTS==0.5.0) (2.4)
Requirement already satisfied: torchaudio in /nix/store/yickqfzqrdwa904ip9ny27lqlzslkm3j-python3.9-torchaudio-0.10.0/lib/python3.9/site-packages (from TTS==0.5.0) (0.10.0+cu113)
Requirement already satisfied: flask in /nix/store/lx89vl46dbrizd8i0ndclj9cvc4hbbd0-python3.9-Flask-2.0.2/lib/python3.9/site-packages (from TTS==0.5.0) (2.0.2)
Requirement already satisfied: pyyaml in /nix/store/qpxjvij39nrs1zaha9fsb8sbcpa90w7g-python3.9-PyYAML-6.0/lib/python3.9/site-packages (from TTS==0.5.0) (6.0)
Requirement already satisfied: fsspec>=2021.04.0 in /nix/store/xhq2pvw5jx4yycamhi7im9119s7zaggn-python3.9-fsspec-2021.10.1/lib/python3.9/site-packages (from TTS==0.5.0) (2021.10.1)
Requirement already satisfied: unidic-lite==1.0.8 in /nix/store/j2hxcfpjs3swx0lj3cwrgnbjzai0v3fn-python3.9-unidic-lite-1.0.8/lib/python3.9/site-packages (from TTS==0.5.0) (1.0.8)
Requirement already satisfied: pypinyin in /nix/store/ckmk3balymwdv1yj2hnbq38v97x90v84-python3.9-pypinyin-0.44.0/lib/python3.9/site-packages (from TTS==0.5.0) (0.44.0)
Requirement already satisfied: scipy>=0.19.0 in /nix/store/62ni5xy88b90mlv94jpdd2dmwwn99b6j-python3.9-scipy-1.7.3/lib/python3.9/site-packages (from TTS==0.5.0) (1.7.3)
Requirement already satisfied: librosa in /nix/store/y9rrx6yvnmyr45ggqzzg5y2nj0v0wz02-python3.9-librosa-0.8.1/lib/python3.9/site-packages (from TTS==0.5.0) (0.8.1)
Requirement already satisfied: gruut in /nix/store/05z3nn3j1h4bz2ing5z0w370iik0q613-python3.9-gruut-2.2.0/lib/python3.9/site-packages (from TTS==0.5.0) (2.2.0)
Requirement already satisfied: pysbd in /nix/store/426napc4vmvpvbqa8zvkk3xgwp7kw3a2-python3.9-pysbd-0.3.4/lib/python3.9/site-packages (from TTS==0.5.0) (0.3.4)
Requirement already satisfied: umap-learn in /nix/store/b0hhymss0lsf91ws108f0fvcdk5sxn9d-python3.9-umap-learn-0.5.2/lib/python3.9/site-packages (from TTS==0.5.0) (0.5.2)
Requirement already satisfied: anyascii in /nix/store/f9x1zaqxmpspajy7qaxh676jfp79naxv-python3.9-anyascii-0.3.0/lib/python3.9/site-packages (from TTS==0.5.0) (0.3.0)
Requirement already satisfied: click>=7.1.2 in /nix/store/nckfzfxmw7ywdzzsp1h06w8k94ryc7m5-python3.9-click-8.0.3/lib/python3.9/site-packages (from flask->TTS==0.5.0) (8.0.3)
Requirement already satisfied: Jinja2>=3.0 in /nix/store/50x9wnqbggbkfcli4c6y4hrflli2xi9d-python3.9-Jinja2-3.0.3/lib/python3.9/site-packages (from flask->TTS==0.5.0) (3.0.3)
Requirement already satisfied: itsdangerous>=2.0 in /nix/store/03x4ryw83lbjy2wbkpwi0lcp2x6x61fi-python3.9-itsdangerous-2.0.1/lib/python3.9/site-packages (from flask->TTS==0.5.0) (2.0.1)
Requirement already satisfied: Werkzeug>=2.0 in /nix/store/mnxcjiwcm6yi8507z6vklyw4fi5c9hvg-python3.9-werkzeug-2.0.2/lib/python3.9/site-packages (from flask->TTS==0.5.0) (2.0.2)
Requirement already satisfied: filelock in /nix/store/pk4a1dl1gcsmxa3nvpkgzy6m9yrfg8vg-python3.9-filelock-3.3.0/lib/python3.9/site-packages (from gdown->TTS==0.5.0) (3.3.0)
Requirement already satisfied: beautifulsoup4 in /nix/store/n6zh9akxns6las8xxlnlww5gbx1fllxz-python3.9-beautifulsoup4-4.10.0/lib/python3.9/site-packages (from gdown->TTS==0.5.0) (4.10.0)
Requirement already satisfied: requests[socks] in /nix/store/rig6npd9sd45ashf6fxcwgxzm7m4p0l3-python3.9-requests-2.26.0/lib/python3.9/site-packages (from gdown->TTS==0.5.0) (2.26.0)
Requirement already satisfied: six in /nix/store/1dm9ndgg56ylawpcbdzkhl03fg6777rr-python3.9-six-1.16.0/lib/python3.9/site-packages (from gdown->TTS==0.5.0) (1.16.0)
Requirement already satisfied: jsonlines~=1.2.0 in /nix/store/hd90xzdsb28x4bi30i5n232bg00mnl1c-python3.9-jsonlines-1.2.0/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (1.2.0)
Requirement already satisfied: num2words<1.0.0,>=0.5.10 in /nix/store/0xzwkb6mih125p6klfsv38y079x233nr-python3.9-num2words-0.5.10/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (0.5.10)
Requirement already satisfied: gruut-lang-en in /nix/store/103ff6sx3paxr3gcxpy7gmfd8722pl5s-python3.9-gruut-lang-en-2.2.0/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (2.0.0)
Requirement already satisfied: python-crfsuite~=0.9.7 in /nix/store/120569ppkm5s2hmyax69gbc13jchv55m-python3.9-python-crfsuite-0.9.7/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (0.9.7)
Requirement already satisfied: gruut-ipa~=0.12.0 in /nix/store/gv5f5pdgjrk0lgwkpg6vvqyb7k1c7gy2-python3.9-gruut-ipa-0.12.0/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (0.12.0)
Requirement already satisfied: networkx<3.0.0,>=2.5.0 in /nix/store/fjmhgdjrvc7q7y3y56lhc800rj3vkgi4-python3.9-networkx-2.6.3/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (2.6.3)
Requirement already satisfied: dateparser in /nix/store/w64g1k95ri9n6frjiziqdvhy9gdf6mv4-python3.9-dateparser-1.1.0/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (1.1.0)
Requirement already satisfied: Babel<3.0.0,>=2.8.0 in /nix/store/rkghmghfr3jxf2wlsys1gnvgdxbpk6pg-python3.9-Babel-2.9.1/lib/python3.9/site-packages (from gruut->TTS==0.5.0) (2.9.1)
Requirement already satisfied: decorator>=3.0.0 in /nix/store/prq5gfwzs8sy5w0758c3j2p4ahiql3j7-python3.9-decorator-5.1.0/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (5.1.0)
Requirement already satisfied: joblib>=0.14 in /nix/store/r3gi7hfh3lxaf4di3jw8cxx72nwk5zrh-python3.9-joblib-1.1.0/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (1.1.0)
Requirement already satisfied: audioread>=2.0.0 in /nix/store/fj1a24x0lx48v3n0hxbf2sgn3aaq5p59-python3.9-audioread-2.1.9/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (2.1.9)
Requirement already satisfied: scikit-learn!=0.19.0,>=0.14.0 in /nix/store/5vfvk37mbyfx4qxcz8frvpvxp42f8hxd-python3.9-scikit-learn-1.0.2/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (1.0.2)
Requirement already satisfied: packaging>=20.0 in /nix/store/gy3pnc7bpff1h4ylhrivs4cjlvmxl0dk-python3.9-packaging-20.9/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (20.9)
Requirement already satisfied: resampy>=0.2.2 in /nix/store/fc9dzlhfkmvgwvwqx2z8lkxdmhi0b5rk-python3.9-resampy-0.2.2/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (0.2.2)
Requirement already satisfied: pooch>=1.0 in /nix/store/9gj4brr04j0wqsvnhqalr14k4fdp6135-python3.9-pooch-1.5.2/lib/python3.9/site-packages (from librosa->TTS==0.5.0) (1.5.2)
Requirement already satisfied: setuptools in /nix/store/w1lsr2i37fr0mp1jya04nwa5nf5dxm2n-python3.9-setuptools-57.2.0/lib/python3.9/site-packages (from numba->TTS==0.5.0) (57.2.0.post0)
Requirement already satisfied: llvmlite<0.38,>=0.37.0rc1 in /nix/store/bm91icwn447r4h6g9mj3mi03wdirlw4k-python3.9-llvmlite-0.37.0/lib/python3.9/site-packages (from numba->TTS==0.5.0) (0.37.0)
Requirement already satisfied: cffi>=1.0 in /nix/store/ig2bdwmplvs6dyg07fdyh006ha768jh1-python3.9-cffi-1.15.0/lib/python3.9/site-packages (from soundfile->TTS==0.5.0) (1.15.0)
Requirement already satisfied: cycler>=0.10 in /nix/store/w1isgqzffs2pzj9312mbjgk8qjnvbcih-python3.9-cycler-0.11.0/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (0.11.0)
Requirement already satisfied: fonttools>=4.22.0 in /nix/store/5fism0m4a19lqla9nx6i58fzfkr318yk-python3.9-fonttools-4.26.2/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (4.26.2)
Requirement already satisfied: python-dateutil>=2.7 in /nix/store/prbmd3abzxry9rh6gc8aclcdszs1jd0s-python3.9-python-dateutil-2.8.2/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /nix/store/igmbvlfi6n7gxf8yf3xa8jgp5jnh3lk3-python3.9-kiwisolver-1.3.2/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /nix/store/803ffb21rv4af521pplb72zjm1ygm9kk-python3.9-pyparsing-2.4.7/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (2.4.7)
Requirement already satisfied: pillow>=6.2.0 in /nix/store/gdk7w7mj34r2jifl483hpz65kcfh27di-python3.9-Pillow-8.4.0/lib/python3.9/site-packages (from matplotlib->TTS==0.5.0) (8.4.0)
Requirement already satisfied: pytz>=2017.3 in /nix/store/zbp0p651wxfkj9l3800zk425dmf7ad3j-python3.9-pytz-2021.3/lib/python3.9/site-packages (from pandas->TTS==0.5.0) (2021.3)
Requirement already satisfied: protobuf>=3.8.0 in /nix/store/yvrxljpavz1rrqn19qs654abj4ckd00s-python3.9-protobuf-3.19.0/lib/python3.9/site-packages (from tensorboardX->TTS==0.5.0) (3.19.0)
Requirement already satisfied: typing-extensions in /nix/store/j3nf4jlx6dvz73r3vby641faavkbp2xl-python3.9-typing_extensions-<IP_ADDRESS>/lib/python3.9/site-packages (from torch->TTS==0.5.0) (<IP_ADDRESS>)
INFO: pip is looking at multiple versions of torch to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of tensorboardx to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pyyaml to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pyworld to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pysbd to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pypinyin to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of pandas to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of mecab-python3 to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of matplotlib to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of soundfile to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of numba to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of librosa to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of jieba to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of inflect to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of gruut to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of gdown to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of flask to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of cython to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of coqpit to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of anyascii to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of numpy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of scipy to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of fsspec to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of unidic-lite to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of tts to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install torchaudio==0.10.0+cu113 and tts==0.5.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    tts 0.5.0 depends on torch
    torchaudio 0.10.0+cu113 depends on torch==1.10.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
builder for '/nix/store/19mvg20hv2scq1ywn1d3x2kqqnmq53f6-tts-0.5.0.drv' failed with exit code 1

Builds for me, untested. Please review.

Yep, tv still sounds similar weird: https://dl.thalheim.io/TSlTPnBA9wOPWpuYkhCfWA/tv.wav

@Mic92 they don't consider that a bug :grimacing:
https://github.com/coqui-ai/TTS/issues/484

@davidak I am now just using a different model: https://github.com/Mic92/dotfiles/blob/master/nixos/eve/modules/tts.nix#L7


qa_openstack: start virtlogd service if available.

Meanwhile a new ISO arrives, this workaround can help to move to green cleanvm for SLE12SP2

Theres no need to get cleanvm Green.gating worka without sle12sp2 and this will hide the Problem if it reappears in sp3

@toabctl I understand this point and I am in favor of reverting it as soon this gets fixed in SP2 and in out images, but I we have this code in mkcloud, and we want libvirt running properly, so tempest can pass (or not).
This tempest test is the real goal for cleanvm, and not to detect problems in SP2 and SP3 (but again, I agree in reverting it)

@toabctl true
This PR was meant as temporary workaround. Maybe this should be reflected in the comment.
Are you fine if we open the revert PR right after merging this? So we will not forget to revert it.
@aplanas
Are you fine with this as well? Or should we skip this PR entirely?

@jdsn We can do this, yes.

:) same idea at the same time - ok

Revert commit sounds good...


asyncproducer fails with topic does not exist when consecutive messages are published to different topics in the same millisecond

Versions

Sarama
Kafka
Go

1.37.2
2.0.0
1.20.1

Configuration
func ProducerConfig() *sarama.Config {
	conf := sarama.NewConfig()
	conf.Producer.Compression = sarama.CompressionSnappy
	conf.Producer.RequiredAcks = sarama.WaitForAll
	conf.Producer.Return.Successes = true
	conf.Producer.Return.Errors = true
	conf.Producer.Retry.Max = 5
	conf.Producer.Retry.Backoff = 500 * time.Millisecond
	conf.Metadata.Full = false
	conf.Metadata.Retry.Max, _ = 50
	conf.Metadata.Retry.Backoff = time.Duration(100) * time.Millisecond
	conf.ClientID = "clientID"
	conf.Version = sarama.V2_0_0_0
	return conf
}

Logs
When I enable logs to print directly to stdout it takes few ms so the issue is not reproducible so I have the logger to print to stdout using a goroutine, so the below logs lines might not be in the exact order.
I updated source code to print stacktrace at places where all ErrUnknownTopicOrPartition was thrown
The below test uses a consumer to consume from 2 topics and a async producer to publish to 2 topics
logs: CLICK ME

=== RUN   TestRoundTrip
Successfully initialized new client
Initializing new client
Initializing new client
Successfully initialized new client
Initializing new client
Successfully initialized new client
Initializing new client
Successfully initialized new client
Initializing new client
Successfully initialized new client
Initializing new client
Initializing new client
Consumer.MaxWaitTime is very low, which can cause high CPU and network usage. See documentation for details.
Deprecation warning: Consumer.Offsets.CommitInterval exists for historical compatibility and should not be used. Please use Consumer.Offsets.AutoCommit, the current value will be ignored
Successfully initialized new client
Successfully initialized new client
Initializing new client
Successfully initialized new client
client/metadata fetching metadata for [test_rn.pos_transaction.pb test_rn.pos_transaction_historical.pb] from broker kafka:9092
Consumer.MaxWaitTime is very low, which can cause high CPU and network usage. See documentation for details.
Deprecation warning: Consumer.Offsets.CommitInterval exists for historical compatibility and should not be used. Please use Consumer.Offsets.AutoCommit, the current value will be ignored
Connected to broker at kafka:9092 (unregistered)
client/brokers registered new broker #0 at 44fe8f289aff:29092Consumer.MaxWaitTime is very low, which can cause high CPU and network usage. See documentation for details.
Deprecation warning: Consumer.Offsets.CommitInterval exists for historical compatibility and should not be used. Please use Consumer.Offsets.AutoCommit, the current value will be ignored
Connected to broker at 44fe8f289aff:29092 (registered as #0)
client/metadata fetching metadata for [test_rn.pos_transaction.pb test_rn.pos_transaction_historical.pb] from broker kafka:9092
client/coordinator requesting coordinator for test-ab1f9599-adae-11ed-8049-80008c44297f-default-worker from kafka:9092
client/coordinator coordinator for test-ab1f9599-adae-11ed-8049-80008c44297f-default-worker is #0 (44fe8f289aff:29092)
client/coordinator requesting coordinator for test-ab1f9599-adae-11ed-8049-80008c44297f-default-worker from kafka:9092
client/coordinator coordinator for test-ab1f9599-adae-11ed-8049-80008c44297f-default-worker is #0 (44fe8f289aff:29092)
Successfully initialized new client
Initializing new client
client/metadata fetching metadata for [test_rn.pos_transaction.pb] from broker kafka:9092
goroutine 179222 [running]:
runtime/debug.Stack()
        /usr/local/go/src/runtime/debug/stack.go:24 +0x64
runtime/debug.PrintStack()
        /usr/local/go/src/runtime/debug/stack.go:16 +0x1c
github.com/Shopify/sarama.(*client).Partitions(0x0?, {0x4006b660c0, 0x25})
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/client.go:348 +0xb4
github.com/Shopify/sarama.(*topicProducer).partitionMessage.func1()
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/async_producer.go:534 +0x98
github.com/eapache/go-resiliency/breaker.(*Breaker).doWork.func1(0x40062b0678?, 0x177a4?)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/eapache/go-resiliency@v1.3.0/breaker/breaker.go:85 +0x54
github.com/eapache/go-resiliency/breaker.(*Breaker).doWork(0x0?, 0x0, 0x0?)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/eapache/go-resiliency@v1.3.0/breaker/breaker.go:86 +0x30
github.com/eapache/go-resiliency/breaker.(*Breaker).Run(...)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/eapache/go-resiliency@v1.3.0/breaker/breaker.go:55
github.com/Shopify/sarama.(*topicProducer).partitionMessage(0x4007d9fe40?, 0x4005c70b40)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/async_producer.go:525 +0x90
github.com/Shopify/sarama.(*topicProducer).dispatch(0x4007d9fe40)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/async_producer.go:502 +0x60
github.com/Shopify/sarama.withRecover(0x0?)
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/utils.go:43 +0x40
created by github.com/Shopify/sarama.(*asyncProducer).newTopicProducer
        /Users/dinesh.udayakumar/go/pkg/mod/github.com/!shopify/sarama@v1.37.2/async_producer.go:495 +0x1fc
err=kafka: Failed to produce message to topic test_rn.pos_transaction_historical.pb: kafka server: Request was for a topic or partition that does not exist on this broker
client/metadata fetching metadata for [test_rn.pos_transaction_historical.pb] from broker kafka:9092
Flush Start
consumer/broker/0 accumulated 6 new subscriptions
consumer/broker/0 added subscription to test_rn.pos_transaction_historical.pb/0
consumer/broker/0 added subscription to test_rn.pos_transaction.pb/2
consumer/broker/0 added subscription to test_rn.pos_transaction_historical.pb/1
consumer/broker/0 added subscription to test_rn.pos_transaction.pb/0
consumer/broker/0 added subscription to test_rn.pos_transaction_historical.pb/2
consumer/broker/0 added subscription to test_rn.pos_transaction.pb/1
Connected to broker at kafka:9092 (unregistered)
client/brokers registered new broker #0 at 44fe8f289aff:29092producer/broker/0 state change to [open] on test_rn.pos_transaction.pb/2
producer/broker/0 starting up
Connected to broker at 44fe8f289aff:29092 (registered as #0)
    utility.go:61: lightspeed_fetcher_test.go:314 Got error: kafka producer errors: [kafka: Failed to produce message to topic test_rn.pos_transaction_historical.pb: kafka server: Request was for a topic or partition that does not exist on this broker]
--- FAIL: TestRoundTrip (0.15s)

Problem Description
We are using the AsyncProducer to publish messages to Kafka and in one of our test cases we are creating the AsyncProducer and publishing multiple messages and intermittently we see that the publish fails with topic/partition does not exist on this broker. By looking at the Kafka request logs and the sarama source code, I could identify that when the first 2 messages getting published are sent at the same millisecond the refreshMetadata method is returning nil and paritionMessage method is throwing the error.
This could be due to the below line in method tryRefreshMetadata in client.go
		t := atomic.LoadInt64(&client.updateMetaDataMs)
		if !atomic.CompareAndSwapInt64(&client.updateMetaDataMs, t, time.Now().UnixNano()/int64(time.Millisecond)) {
			return nil
		}

@dineshudayakumar please can you test with latest sarama to see if the concurrency fixes have resolved this issue?

Sure, will test and confirm in a day or two (and thanks for fixing)

Looks like I am not seeing this issue anymore, happy to close it. Thank you!!


After search, adjusting marker

Hi there,
I'm trying to establish if after a location search, user can then drag the marker to adjust the location? I'm using leaflet.js with openlayers. Here is example: http://pastpin.com/photo/45508949092. User would search 'sydney' then zoom map. I cannot figure out how to let user then move the marker to best position before save.
Many thanks for help!
Neil

First of all, this sounds like a stack overflow question, not an issue.
Secondly, why are you using both leaflet.js and openlayers? You only need one map client.
You have to have a service available where you can search for a name and get coordinates for the results. Then you need to place a marker in the map (use this as an example). If you want to edit the feature after it's placed on the map, you can use the modify interaction (use this as an example).

Please ask usage questions on stackoverflow instead with the tag openlayers
https://stackoverflow.com/questions/tagged/openlayers


Remove the extra hack from the Graphics.CopySource operation

We should be able to look up the drawable when we perform the operation instead.

Nice to have. Pushing to 'Future'.


feat: detect frequently restarting pods

resolves: https://github.com/flanksource/is-healthy/issues/70


s3.PutObject always returns NotImplemented

Because Go's http client implicitly adds a "Transfer-Encoding: chunked" header when the request body is not nil and the request ContentLength <= 0, PutObject will always return an error.
This can be worked around by defining a new http.RoundTripper to be used as the Transport in the http.Client passed to s3.New that will read the body and add the ContentLength field, but this should be handled by the SDK.

The SDK will be adding ContentLength when possible (i.e., for seekable streams), but it won't always be possible, in which case ContentLength can be provided manually. Note that you currently can provide a ContentLength to PutObject, which should allow S3 to work currently.

The above commit should improve the logic for the SDK building Content-Length on seekable reader objects. Non-seekable streams will still require a Content-Length to be provided, but that will actually cause an error to be thrown prior to sending, so the Transfer-Encoding behavior should not occur in develop (and in master, once the branch is merged).

Related to #139


Replacing Commander Spellbook Bulk Api with HTTP queries

In order to fetch the data in real time without relying on a bulk json asset shipped with the frontend files an HTTP REST API should be designed for:

Combo/Variant search use case
Find My Combos use case (issue #343)

solved in next big release


Release 0.5.0

Tracking the checklist for completing release 0.5.0

[ ] Release Notes (feature highlights)
[ ] Documentation for new features
[ ] Upgrade story
[ ] Performance Results

I linked #166 here - this issue is to clean up API code to make sure we have a clean API for 0.5.0 release.

assigned to @zhaijack since he is the release manager for 0.5.0

close it


Add read time metrics for client local cache

What changes are proposed in this pull request?
Add two metrics for tracking read IO time concerning client cache.
Why are the changes needed?
Collect statistics about read IO time in cases of cache hits and misses, to provide insights about local cache performance.
Does this PR introduce any user facing changes?
No.

Codecov Report

Merging #13829 (9eecb1d) into shadow_cache_improve (014d108) will decrease coverage by 12.81%.
The diff coverage is 0.00%.

@@                     Coverage Diff                     @@
##             shadow_cache_improve   #13829       +/-   ##
===========================================================
- Coverage                   32.82%   20.00%   -12.82%
+ Complexity                   6542      963     -5579
===========================================================
  Files                        1386      558      -828
  Lines                       80572    33904    -46668
  Branches                     9806     4450     -5356
===========================================================
- Hits                        26448     6784    -19664
+ Misses                      51825    26794    -25031
+ Partials                     2299      326     -1973

Impacted Files
Coverage Δ

...ommon/src/main/java/alluxio/metrics/MetricKey.java
0.00% <0.00%> (-97.65%)
:arrow_down:

...common/src/main/java/alluxio/grpc/GrpcService.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...common/src/main/java/alluxio/time/TimeContext.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mmon/src/main/java/alluxio/metrics/MetricInfo.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mmon/src/main/java/alluxio/time/ThreadSleeper.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mon/src/main/java/alluxio/concurrent/LockMode.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mon/src/main/java/alluxio/shell/CommandReturn.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mon/src/main/java/alluxio/util/ExceptionUtils.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...mon/src/main/java/alluxio/util/WaitForOptions.java
0.00% <0.00%> (-100.00%)
:arrow_down:

...n/src/main/java/alluxio/wire/AlluxioProxyInfo.java
0.00% <0.00%> (-100.00%)
:arrow_down:

... and 1141 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 014d108...9eecb1d. Read the comment docs.

@beinan @dbw9580
thanks for the improvement. left some comments

Fixed most of the code review issue,  @apc999 could you take a second look? thanks!

alluxio-bot, merge this please


Repurpose coupled_cross_entropy_norm and its related functions

This function provides numerical efficiency in computing the coupled cross entropy for the coupled Gaussian
Change name to coupled_cross_entropy_coupled_gaussian
Change name of coupled_cross_entropy to coupled_cross_entropy_general.
Create wrapper function which calls the subfunctions coupled_cross_entropy_coupled_gaussian and coupled_cross_entropy_general
The documentation at the head of the function should refer to the appropriate equations.
The same change needs to be completed for the coupled_entropy_norm and the coupled_divergence_norm

A better description of this issue was added. See "Treating special cases within the coupled entropy function". Closing this issue.


Bugfix for controller tests that support Session Scope in controllers

Currently session (and request) scope is not supported in controller tests

This change is

Coverage increased (+0.2%) to 69.91% when pulling b7100d4d2e2e0760e8fa83c5fe9239f821176650 on scopes-in-test-bugfix into 57c722c9834a80c9a08cbea136ccf4c65e0244fd on master.

Reviewed 9 of 9 files at r1.
Review status: all files reviewed at latest revision, all discussions resolved.

Comments from Reviewable


Enhance the UI of the Write blog page

hey! i want to work on this issue under jwoc

Hey @AtriSukul1508 , I would like to work on this issue under JWOC. Can you please assign it to me?

Assigned @Tushar-go, make sure to attach a screenshot while creating a PR and maintain the current color combination

@Tushar-go any updates?


Unable to install ballerina (ballerina-macos-installer-x64-1.2.10.pkg) on Mac OS Big Sur

Description:

Unable to install ballerina (ballerina-macos-installer-x64-1.2.10.pkg) on Mac OS Big Sur. It reports the following error when double-clicking it to install:
“ballerina-macos-installer-x64-1.2.10.pkg” can’t be opened because Apple cannot check it for malicious software.
This software needs to be updated. Contact the developer for more information.
Suggested Labels:

Suggested Assignees:

Affected Product Version:
OS, Browser, other environment details and versions:
Steps to reproduce:
Related Issues:

We are working on it by notarizing the installer with Apple.
Here its not possible to continue the installation as there is only "OK" button as below

However there is a temporary workaround in mac where you need to right click and click open. Then you will get "Open" button as below

This has been fixed from swan lake beta1 onward by notarizing the installers


Installation fails

The installation files hosted at speechkitchen.org are no longer available. Is there a mirror site or alternatives to get these files?

speech-kitchen.org should work. please let me know if not.


Pull in TeraMath compatibility, remove engine's Vector3i

Use the newest version of TeraMath from https://github.com/MovingBlocks/TeraMath/pull/26
Rename org.terasology.math.Vector3i into org.terasology.math.Vector3iUtil and make all its unique methods static.
I have a large number of the modules in my working copy as well and will make appropriate PRs on those repos as well in the next day or so.

Cervator,  Would you be able to walk me through the process of using the new TeraMath library after it is merged?  Likely its just updating a version number someplace, right?

Yeah TeraMath isn't fully set up the "formal" way when it comes to releases.
There is a version tag in https://github.com/MovingBlocks/TeraMath/blob/develop/gradle.properties you can increment either in your PR or after merging it.
http://jenkins.terasology.org/view/Libs/job/TeraMath/ should build automatically after the merge and will publish a snapshot to Artifactory's libs-snapshot-local (as said no release / non-snapshot setup is live yet)
After a new version is in Artifactory the engine should pick it automatically due to https://github.com/MovingBlocks/Terasology/blob/develop/engine/build.gradle#L124 still being configured to using the very latest version it can find
Really we should make a proper release job that makes non-snapshots and changing the engine dependency to a hard-set version, but that can wait till after this sometime. I want to go over the release-needing jobs in Jenkins soon enough :-)
@msteiger + @skaldarnar either or both of you should probably test/merge this. If you review and are happy with it yet want me to run it through my mega-workspace to catch any broken modules let me know and I can do that.

Light And Shadow is remaining.  I accidentally mixed the change in with something else.

Yes,  I can confirm that it automatically picks up the new TeraMath library.  Thanks for merging that @msteiger.

So should we start clicking buttons or doing Git stuff manually to get this PR + friends merged? :-)

All this looks good from my perspective.  I have made PRs from local branches in the the modules I had in my workspace to hopefully make this process easier.  @Cervator,  maybe you would be best suited to merge the modules and then i[date them in your mega workspace and fix the few strays that have been missed.  Or maybe it might be easier just to do the module fixes yourself and commit?  Dunno.  It is just a lot of noise no matter the angle you look at it.

Yep I can do the main PR merge then grab all the module PRs and finally catch any stragglers. Just need to find some time for it.

Test this please
(Yay the PR builder plugin is working!)

Refer to this link for build results (access rights to CI server needed):
http://jenkins.terasology.org/job/TerasologyPRs/2/
Jenkins Success Message

Finally got through this and all the modules! Thanks for priming some of them, especially with a branch instead of a fork, made it a fair bit easier :-)
28 modules changed in total. Notification spam inc!

Hmm, trickier than I thought! Local builds are fine. Module builds in Jenkins break
Been testing with AnotherWorld: http://jenkins.terasology.org/view/Modules/job/AnotherWorld/64/console
This has happened before where a newer snapshot has been reluctant to start resolving in Jenkins, but I outright deleted the entire Gradle cache on the server and added --refresh-dependencies to the module build template. I also wiped my local Gradle cache in case I had a false positive on things looking good. I also rebuilt the engine in Jenkins after the cache wipe and verified the newly pulled TeraMath looked to be the right version.
I noticed the TeraMath.calcBlockPos in BeachDecorator is deprecated, as is what it uses in the engine TeraMath wrapper.
But still, it works in IntelliJ locally, but not in Jenkins. What gives? I'm confused and sleepy, out of time for tonight :-)

Found it after another evening sunk into fun triage - the Artifactory plugin for Jenkins fixed a bug we have been getting a free ride on for ages ;-)
I updated the plugins as part of the dynamic builder setup. Didn't matter at first since the couple PRs I merged were inconsequential when related to dependency resolution.
TeraMath/Vector3i change hit - boom! The module jobs were suddenly respecting the resolution repo setting for where to get dependencies, and with their correct config weren't pointing to the right place to get the latest engine dependency (or so - TeraMath itself seemed fine)
So now we can finally do it right, but at nearly 1 am I'm not awake enough or staying around long enough to fix. But at least I know what to fix now ;-)


The remove function doesn't work.

Even if you type /remove 1, track 2 will be removed instead of track 1.

i think you should do /remove 0 since what you consider track 1, the bot will consider it as track 0

There might be a bug with this but we will fix it asap!

Even if you type /remove 1, track 2 will be removed instead of track 1.

Because 0 is the first entry of a list


Update links in README.md

Summary
Originally posted by @svelle here : https://community.mattermost.com/core/pl/uos187ni87g63ekp6wbn4sj8ro

Hello @haardikdharma10,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.


SQLExpressions.unionAll returns "No connection provided"

using querydsl version 4.1.4
The connection info is not filled in when using SQLExpressions.unionAll.
JPAQueryFactory queryFactory = new JPAQueryFactory(entityManager);
JPAQuery jpaQuery1 = queryFactory.select(Expressions.constant("abc"), QAlertOverview.alertOverview.alertId.count()).from(QAlertOverview.alertOverview);
JPAQuery jpaQuery2 =  queryFactory.select(Expressions.constant("def"), QAlert.alert.id.count()).from(QAlert.alert);
List list1 = jpaQuery1. fetch();
List list2 = jpaQuery2. fetch();
List list1and2 = SQLExpressions.unionAll(jpaQuery1, jpaQuery2).fetch();

The fetch of list1 and list2 work correctly while fetch of list1and2 throw "java.lang.IllegalStateException: No connection provided", which is logical since SQLExpressions.unionAll does not copy connection info;
What's the correct way of executing the unionized query?
Thanks!

huh. Interesting that we never found this.
This indeed shouldn't work.
The process should be like this.
So you would need a query with connection information, and call union or unionAll and work from there.
@querydsl/developers I think it's best if we remove these methods, don't you think?
Would it be reasonable to just remove them (since they don't work) or remove them with Querydsl 5 or so?


Update migrations

When you change the structure of the database, you should create new migration for updates, rather than modifying the existing migration.

Hi! First of all thank you that paid attention for our module.
As I mentioned on www.yee-soft.com YeeCMS is currently under active development. Currently multilingual site is being developed. We are preparing to release 0.1 version of CMS.  After that we will fully support migrations.
Sorry for the inconvenience.

OK.  Thank you for all this great work.  Yee CMS looks promising.  I am testing it intensively.


Issues with -gamepadui in beta client

Expected Behavior
The current Steam Beta can be launched with the commandline -gamepadui to use the Steam Deck UI on WIndows. When running in this mode, Steam doesn't seem to fully pick up the process as active: the cursor remains on the busy cursor, and the bar at the bottom stays on screen as the game plays. Hitting the Steam button pops up the Steam overlay as expected.
Still seems to work if I launch Steam normally, and use the old Big Picture.
Steps to Reproduce the Problem

Update to Steam Beta client
Launch Steam with steam.exe -gamepadui
Launch game (UWP from Game Pass in this case, TMNT Shredder's Revenge)

General Information

Version: Latest release as of this post
Platform: Win11 Deck
Controller(s): Deck controller
Application: GlosSITarget

Logs / Additional Files
Toggle me!
[2022-10-29 18:25:19.184] [log] [info] GlosSITarget version: <IP_ADDRESS>
[2022-10-29 18:25:19.189] [log] [debug] Read config file "C:\Users\Mike\AppData\Roaming\GlosSI\Targets\Teenage Mutant Ninja Turtles Shredder's Revenge.json"; config: {"controller":{"allowDesktopConfig":false,"emulateDS4":false,"maxControllers":1},"devices":{"hideDevices":true,"realDeviceIds":false},"extendedLogging":false,"icon":"C:\\Program Files\\WindowsApps\\DotEmu.TeenageMutantNinjaTurtlesShreddersRevenge_1.2207.8.0_x64__map6zyh9ym1xy\\Square150x150Logo.png","launch":{"closeOnExit":true,"ignoreLauncher":true,"killLauncher":false,"launch":true,"launchAppArgs":null,"launchPath":"DotEmu.TeenageMutantNinjaTurtlesShreddersRevenge_map6zyh9ym1xy!TMNT","launcherProcesses":[],"waitForChildProcs":true},"name":"Teenage Mutant Ninja Turtles: Shredder's Revenge","snapshotNotify":false,"version":1,"window":{"disableOverlay":false,"maxFps":null,"scale":null,"windowMode":false}}
[2022-10-29 18:25:19.190] [log] [info] Running on Windows 11; Winver: 10.0.22621
[2022-10-29 18:25:19.190] [log] [info] Detected Steam Path: c:/program files (x86)/steam
[2022-10-29 18:25:19.190] [log] [info] Detected Steam UserId: 3288412
[2022-10-29 18:25:19.204] [log] [info] Detected Overlay hotkey(s): Shift+KEY_TAB
[2022-10-29 18:25:19.212] [log] [debug] Connected to ViGEm
[2022-10-29 18:25:19.217] [log] [info] Detected screenshot hotkey(s): KEY_F12
[2022-10-29 18:25:19.594] [log] [info] Detected resolution: 1280x800
[2022-10-29 18:25:19.594] [log] [info] Creating Overlay window (Borderless Fullscreen)...
[2022-10-29 18:25:19.706] [log] [debug] Full screen size: 1280x800
[2022-10-29 18:25:19.706] [log] [debug] Primary monitor size: 1280x800
[2022-10-29 18:25:19.706] [log] [debug] Window position: 0x0
[2022-10-29 18:25:19.706] [log] [info] Resizing window to 1px smaller than fullscreen...
[2022-10-29 18:25:19.710] [log] [debug] Screen DPI: 96
[2022-10-29 18:25:19.710] [log] [trace] Limiting FPS to 60
[2022-10-29 18:25:19.735] [log] [warning] Not applying too low screen scale setting
[2022-10-29 18:25:19.735] [log] [trace] Set process priority to HIGH_PRIORITY_CLASS
[2022-10-29 18:25:19.736] [log] [debug] Unpatching Valve CreateProcess Hooks
[2022-10-29 18:25:19.736] [log] [trace] Patching "CreateProcessW"...
[2022-10-29 18:25:19.736] [log] [trace] Unpatched "CreateProcessW"
[2022-10-29 18:25:19.736] [log] [debug] App launch requested
[2022-10-29 18:25:19.743] [log] [debug] Successfully injected
[2022-10-29 18:25:19.743] [log] [info] Successfully injected UWPOverlayEnablerDLL.dll into explorer.exe
[2022-10-29 18:25:19.743] [log] [info] Steam-overlay detected.
[2022-10-29 18:25:19.743] [log] [warning] Double press Steam- overlay key(s)/Controller button to show GlosSI-overlay
[2022-10-29 18:25:19.758] [log] [debug] Successfully injected
[2022-10-29 18:25:19.758] [log] [info] Successfully injected GlosSIWatchdog.dll into explorer.exe
[2022-10-29 18:25:19.763] [log] [debug] Setting up device hiding...
[2022-10-29 18:25:19.763] [log] [debug] Unpatching Valve HID hooks...
[2022-10-29 18:25:19.763] [log] [trace] Patching "SetupDiEnumDeviceInfo"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "SetupDiEnumDeviceInfo"
[2022-10-29 18:25:19.763] [log] [trace] Patching "SetupDiGetClassDevsW"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "SetupDiGetClassDevsW"
[2022-10-29 18:25:19.763] [log] [trace] Patching "HidD_GetAttributes"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "HidD_GetAttributes"
[2022-10-29 18:25:19.763] [log] [trace] Patching "HidD_GetPreparsedData"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "HidD_GetPreparsedData"
[2022-10-29 18:25:19.763] [log] [trace] Patching "HidD_GetProductString"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "HidD_GetProductString"
[2022-10-29 18:25:19.763] [log] [trace] Patching "HidP_GetButtonCaps"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "HidP_GetButtonCaps"
[2022-10-29 18:25:19.763] [log] [trace] Patching "HidP_GetCaps"...
[2022-10-29 18:25:19.763] [log] [trace] Unpatched "HidP_GetCaps"
[2022-10-29 18:25:19.764] [log] [trace] Patching "HidP_GetData"...
[2022-10-29 18:25:19.764] [log] [trace] Unpatched "HidP_GetData"
[2022-10-29 18:25:19.764] [log] [trace] Patching "HidP_GetUsageValue"...
[2022-10-29 18:25:19.764] [log] [trace] Unpatched "HidP_GetUsageValue"
[2022-10-29 18:25:19.764] [log] [trace] Patching "HidP_GetUsages"...
[2022-10-29 18:25:19.764] [log] [trace] Unpatched "HidP_GetUsages"
[2022-10-29 18:25:19.764] [log] [trace] Patching "HidP_GetValueCaps"...
[2022-10-29 18:25:19.764] [log] [trace] Unpatched "HidP_GetValueCaps"
[2022-10-29 18:25:19.769] [log] [info] Hid Gaming Devices; Enabling Overlay element...
[2022-10-29 18:25:19.769] [log] [info] LaunchApp is UWP, launching...
[2022-10-29 18:25:19.769] [log] [debug] Launching UWP app "DotEmu.TeenageMutantNinjaTurtlesShreddersRevenge_map6zyh9ym1xy!TMNT"; args ""
[2022-10-29 18:25:19.979] [log] [info] Launched UWP Package "DotEmu.TeenageMutantNinjaTurtlesShreddersRevenge_map6zyh9ym1xy!TMNT"; PID: 2072
[2022-10-29 18:25:19.980] [log] [debug] Hooking GetForegroundWindow (in own process)
[2022-10-29 18:25:21.275] [log] [debug] Overlay opened
[2022-10-29 18:25:21.275] [log] [debug] Bring own window to foreground
[2022-10-29 18:25:21.275] [log] [debug] Un-Hooking GetForegroundWindow (in own process)
[2022-10-29 18:25:21.275] [log] [debug] Hooking GetForegroundWindow (in own process)
[2022-10-29 18:25:21.797] [log] [info] Plugged in controller 0, 1; VID: 28de; PID: 28e
[2022-10-29 18:25:23.429] [log] [info] Launched App "" with PID "2072" died
[2022-10-29 18:25:23.437] [log] [trace] Child process "" with PID "1040" died
[2022-10-29 18:27:14.318] [log] [info] Launched App "" with PID "9240" died
[2022-10-29 18:27:14.318] [log] [info] Configured to close on all children exit. Shutting down...
[2022-10-29 18:27:17.340] [log] [debug] Started http-server on port 8756
[2022-10-29 18:27:17.345] [log] [info] Un-hid Gaming Devices

The new BPM is actually not working at all with non-steam shortcuts, so it's a Steam problem first rather than a GlosSI issue at the moment.

The new BPM is actually not working at all with non-steam shortcuts, so it's a Steam problem first rather than a GlosSI issue at the moment.

Sorry, do you mind explaining a bit more what you mean?
For me both the new UI and the standard UI launch the game just fine, the problem is that in both the game does not take focus. In the standard UI you can easily click on the game icon in the Taskbar but you can't in the new UI. Shouldn't Glossi take focus in that case?

Shouldn't GlosSI take focus in that case?

Yes, it should.
Thing is, as it currently stands, the new UI is (sometimes) broken even when not using GlosSI.
Also it's beta and only accessible using a commandline-switch, so there won't be support for the time being.

Also it's beta and only accessible using a commandline-switch, so there won't be support for the time being.

Got it thanks.

Shouldn't GlosSI take focus in that case?
Yes, it should.

Is it fair to raise an issue or feature request for this separately given it impacts the steam stable release?

Shouldn't GlosSI take focus in that case?

Yes, it should.

Sorry, I didn't think before answering.
Actually, no.
The GlosSI window should normally not take focus. It should be completely "transparent".
The only case the GlosSI window should take focus is when the (Steam or GlosSI) overlay is opened.

I think I get what you're saying, I guess what I was trying to say is could GlosSI target force the game being launched to take focus? Or do you think it's a problem with Steam (or Game Pass in my case) if that doesn't happen.


Make action-time-limit's system bounds configurable.

In the same vain the memory limits are settable, this should be decidable per deployment.

PG2 2881 🔵


Enable SRR and fix broken links

Remove mounted checks on the providers to enable SSR
Fix broken links preventing build

🟡 Heimdall Review Status

Requirement
Status
More Info

Reviews

🟡
0/1

Denominator calculation

Show calculation

1 if user is bot
0

1 if user is external
0

From .codeflow.yml

1

Additional review requirements

Show calculation

Max
0

0

From CODEOWNERS

0

Global minimum
0

Max

1

1

1 if commit is unverified
0

Sum
1


bug

Environment Info
kivy: 1.9.1
pygments: 2.1.3
docutils: 0.12
watchdog: 0.8.3
jedi: 0.9.0
gitpython: 2.0.2
six: 1.10.0
kivy-garden: 0.1.4
Platform: Windows-7-6.1.7601-SP1
Python: 2.7.11
Traceback
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\kivy\core\window\window_sdl2.py", line 619, in mainloop
self._mainloop()
File "C:\Python27\lib\site-packages\kivy\core\window\window_sdl2.py", line 362, in _mainloop
EventLoop.idle()
File "C:\Python27\lib\site-packages\kivy\base.py", line 327, in idle
Clock.tick()
File "C:\Python27\lib\site-packages\kivy\clock.py", line 515, in tick
self._process_events()
File "C:\Python27\lib\site-packages\kivy\clock.py", line 647, in _process_events
event.tick(self._last_tick, remove)
File "C:\Python27\lib\site-packages\kivy\clock.py", line 406, in tick
ret = callback(self._dt)
File "C:\Python27\Scripts\designer\nodetree.py", line 109, in _refresh
self._clear_tree(self.tree, self.tree.root)
File "C:\Python27\Scripts\designer\nodetree.py", line 102, in _clear_tree
self._clear_tree(tree, n)
File "C:\Python27\Scripts\designer\nodetree.py", line 102, in _clear_tree
self._clear_tree(tree, n)
File "C:\Python27\Scripts\designer\nodetree.py", line 102, in _clear_tree
self._clear_tree(tree, n)
File "C:\Python27\Scripts\designer\nodetree.py", lin

hi @AriellaJ
Sorry, but the traceback is not complete. Are you able to reproduce this bug and post the full content ?

Hi,
I'm closing this bug report because the traceback is not complete. Please, read the Bug Reporter instruction, a complete report must end with "End of Traceback" in the last line. With this report we are not able to identify what is wrong here.
Thank you.


[Event Request] codeunit 7311 "Whse. Worksheet-Create" - OnCreateWhseWkshLineOnAfterInsert

We would like to add a new event OnCreateWhseWkshLineOnAfterInsert in the following procedure:
local procedure CreateWhseWkshLine(var WhseWkshLine: Record "Whse. Worksheet Line") Created: Boolean
var
    Item: Record Item;
    ItemTrackingMgt: Codeunit "Item Tracking Management";
begin
    with WhseWkshLine do begin
        if "Shelf No." = '' then begin
            Item."No." := "Item No.";
            Item.ItemSKUGet(Item, "Location Code", "Variant Code");
            "Shelf No." := Item."Shelf No.";
        end;
        OnCreateWhseWkshLineOnBeforeInsert(WhseWkshLine);
        if Insert() then begin
            Created := true;
            if ItemTrackingMgt.GetWhseItemTrkgSetup("Item No.") then
                ItemTrackingMgt.InitTrackingSpecification(WhseWkshLine);
            OnCreateWhseWkshLineOnAfterInsert(WhseWkshLine); //A200F0000.Event
        end;
    end;
end;

Event Signature:
[IntegrationEvent(false, false)]
local procedure OnCreateWhseWkshLineOnAfterInsert(var WhseWorksheetLine: Record "Whse. Worksheet Line")
begin
    //A200F0000.Event
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Update NFRs in Developer Guide

Resolve #34. Tagging @LikHern to follow up.

Codecov Report

Merging #39 (7868d11) into master (094f9e5) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #39   +/-   ##
=========================================
  Coverage     72.15%   72.15%
  Complexity      399      399
=========================================
  Files            70       70
  Lines          1232     1232
  Branches        125      125
=========================================
  Hits            889      889
  Misses          311      311
  Partials         32       32

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


missing "prop-types" module issue

Prerequisites

[x] I have read the documentation;
[x] In the case of a bug report, I understand that providing a SSCCE example is tremendously useful to the maintainers.

Description
Using library in a project with react version lower then 15.5 and no prop-types results in a missing module error
Expected behavior
prop-types is listed in dependencies or in peer-dependacies
Actual behavior
prop-types is listed in devDependencies
Version
0.47.0

Ah indeed, could you please update the package.json file to move it accordingly and send a PR? Thanks!

so looks like the issue is fixed in 0.49
closing it, sorry


🐞 CLI: dagger listen --disable-host-read-write fails in v0.9.9

What is the issue?
There is a regression on dagger v0.9.9 with the dagger listen command when the flag --disable-host-read-write is specified. Prior to v0.9.8 this was working correctly. Since v0.9.9 it fails with:
Error: failed to get configured module: failed to get local root path: input: resolve: moduleSource: resolveContextPathFromCaller: failed to stat source root: failed to receive file bytes message: rpc error: code = Unimplemented desc = unknown service moby.filesync.v1.FileSync

If you run this command inside a module it also fails with the same error:
$ dagger mod init --name test --sdk go
$ dagger listen --disable-host-read-write
├ [0.01s] loading module
✘ directory ERROR [0.01s]
▶ directory ▶ host.directory /home/matipan/bin/test
  ✘ upload /home/matipan/bin/test from pop-os (client id: 09oib86lak5rdy6j1tbxf2gbu) ERROR [0.01s]
  ├ [0.00s] transferring /home/matipan/bin/test:
• Engine: 6241366fb45d (version v0.9.7)
⧗ 1.97s ✔ 6 ✘ 3
Error: failed to get loaded module ID: input: resolve: host: directory: host directory /home/matipan/bin/test: no local sources enabled
input: resolve: host: directory: host directory /home/matipan/bin/test: no local sources enabled

Dagger version
dagger v0.9.9 (registry.dagger.io/engine) linux/amd64
Steps to reproduce
$ dagger listen --progress plain --disable-host-read-write

Log output
Connected to engine 5c478db0e017 (version v0.9.9)

Error: failed to get configured module: failed to get local root path: input: resolve: moduleSource: resolveContextPathFromCaller: failed to stat source root: failed to receive file bytes message: rpc error: code = Unimplemented desc = unknown service moby.filesync.v1.FileSync

This is likely caused by the changes to FileSync in https://github.com/dagger/dagger/pull/6575.

I am wondering if instead of fixing this, we should stop using listen altogether in the Playground, and maybe remove this feature. dagger listen came up a few times times in the past, and I'm still not sure why we have it:

https://github.com/dagger/dagger/issues/4319
https://github.com/dagger/dagger/issues/5205

The last thread is particularly interesting, since it offers a bunch of alternatives to dagger listen. As that conversation seems to be long-term and filled with bike shedding, I would rather focus on something pragmatic for the Playground.
As we are consolidating everything on EKS, will dagger listen still make sense? I am thinking:

A dedicated node pool for Playground
Playground Engines will become just a DaemonSet
One of the containers in that DaemonSet will be the API

This ensures that both change as a single unit
Liveness checks will ensure that the DaemonSet will not be rolled if there is some sort of incompatibility between the API and the Engine (they are a single unit)
Alternatively, we do not do in-place updates, but deploy new versions in parallel (I'm personally a big fan of this)

API container connects to the Dagger Engine via a unix socket, no TCP port (they are a single unit)
The Web instance selects which version gets used by passing an X-Dagger-Version header - nice trick @matipan 🙌

If we went in the direction described above, would we need dagger listen @marcosnils ?

I use dagger listen sometimes to inspect the API with https://studio.apollographql.com/, for example. I'd be happy with a command that can spin up a playground locally though.

Sounds like a legit reason to keep listen around to me! I actually wasn't aware of it, might have to start using it now.
That said, if we do keep it, it definitely needs a bit of love - including some tests and docs.

I've done a bit of a deep dive into what might be causing this issue.
TL;DR: in #6575 we started to resolve LocalContextPath in the context of the Dagger Engine instead of the host, which requires communicating with the host through gRPC. However, the required services are disabled due to --disable-host-read-write so LocalContextPath cannot be resolved.
Tracking from where the error originates we start in StatCallerHostPath:
func (c *Client) StatCallerHostPath(ctx context.Context, path string, returnAbsPath bool) (*fsutiltypes.Stat, error) {
        // ...
	diffCopyClient, err := filesync.NewFileSyncClient(clientCaller.Conn()).DiffCopy(ctx) // <-- DiffCopy is not available
	if err != nil {
		return nil, fmt.Errorf("failed to create diff copy client: %s", err)
	}
	defer diffCopyClient.CloseSend()
	msg := fsutiltypes.Stat{}
	err = diffCopyClient.RecvMsg(&msg)
	if err != nil {
		return nil, fmt.Errorf("failed to receive file bytes message: %s", err) // <--- This is where it errors out
	}
	return &msg, nil
}

StatCallerHostPath is used by resolveContextPathFromCaller to effectively resolve the LocalContextPath of the module when we are initializing the configuration of a module on all commands (done by optionalModCmdWrapper). This functions are all executed in the context of the Dagger Engine. But since they are trying to resolve paths that are specific to the host they need to make gRPC calls to the buildkit client running in the host to traverse and copy the contents of some directories. However, when DisableHostRW is enabled those services do not get registered:
// filesync
if !c.DisableHostRW {
    bkSession.Allow(AnyDirSource{})
    bkSession.Allow(AnyDirTarget{})
}

How did this used to work in v0.9.8? It seems that both the LocalContextPath (previously called LocalRootPath) and LocalRootSourcePath (previously called LocalSourcePath) were resolved in the context of the CLI. Meaning that no path resolution ever exited the host, so having the FileSync DiffCopy API disabled had no effect.
I'm sure there are good reasons for it, but by just looking at the code I'm not able to understand why LocalContextPath is now resolved by calling the engine. Doing so is what essentially breaks this particular behavior.
Given that I lack this context, I'm not entirely certain what the most appropriate solution would be. If going back to resolve LocalContextPath without exiting the host, adding a fallback to use the host when DisableHostRW is enabled or some other alternative.
I'm happy to keep going, but I think it's worth checking if this all makes sense @jedevc @sipsma

Quick drive-by comment: I think if --disable-host-read-write is passed we can probably just skip any module loading related code. We only use that flag for Playground, which doesn't use modules.
Related, maybe there's a work-around where you can pass -m github.com/foo/bar (literally any module)? That way it'll load a remote module instead of trying to figure out if you're running from a local one.

Yeah for more context, module loading all happens from the engine now because:

It centralized the logic more (module loading previously was split between the cli and engine, now there's only one place to keep track of it all
The module loading logic got way more complex recently due to the need to sparsely load local modules (which multiplies the above point)

The behavior of daggery query is supposed to be that it tries to load a local module if it exists but ignores any error and just operates without the module if it can't load one. I suspect that we just need to ignore an error somewhere. I will take a look at this, assigning to me.


Working example?

It would be really useful for Ruby newbies like me to have a working Rails example if possible :)
I've set up a basic Rails project and used the example code in the Readme, but when I call deliver! on my SparkPost mailer I get an SMTP error:
Net::SMTPAuthenticationError: 535 5.7.8 Sorry.
Complete examples of the steps needed to configure SparkPost Rails would be most welcome!

OK, figured this out, it seems that setting
c.api_key = ENV['SPARKPOST_API_KEY']
is not allowed, removed this line and the mailer works (although unable to get templates working at present)


Parameterise modal background colours

Since my I've designed my app to use the darker colours at the back and the lighter colour at the front, it felt weird that the modals didn't follow the same colour scheme.
It would make sense to have this override behaviour, considering it's available for the lifted tabs (although surprisingly not available for the boxed tabs).

This is not required at all since we can use any custom color for modal or modal-box:
Example: https://play.tailwindcss.com/oBdSOJHq55


Introduced new control-type "watch"

Current situation
When using odrive, i want to have a live-view on some properties. For that, i added a control for each property but noticed, that the controls value is not updated / refreshed. So the "control"-type is only for changing values in odrive, not for displaying states. When using plots for getting current states, enums can not be handled and it is difficult to visualize lots of different properties in single plots...
Suggested and implemented solution
I introduced a new category of controls, called "watch". A watch control is readonly but refreshes its value every second by pulling the value from the odrive. It distinguishes between Boolean, Numeric and Enum types and displays them differently.

Discussion

Is there any option for getting value refreshes other than pulling cyclically? How do the plots get their real-time data?
Has polling a negative influence in e.g. plots performance?
Should the polling interval be adjustable? E.g. axis state may not be changing as rapidly as mabye set_pos will...

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Max seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

I just realized, that now also the Control controls are refreshed / updated. That is probably not very usable, so i would have to remove that behaviour from the Control controls.

Ok, what do you think about that:

Instead of introducing watch types, i enhance to control types as follows:

the actual value of the property is refreshed once a second
if the property has write access, an extra control (input text, select, checkbox) is inserted left of the current value
if this control is changed OR the send button within the two controls is pressed, the value is written down.
you get feedback if writing was successfull whenever the current value has changed

I find that approach quite convenient, because the behaviour is obvious and you can prepare multiple controls with different values for fast switching between two values:

I like this idea! I had considered something similar. For the plots: right now, those are managed separately as "sampled variables". Variables added to the plots are added to store.state.sampledProperties. When sampling is enabled, an event is sent to the server to begin a polling loop at a 20ms period. You can see that in odrive_server.py beginning at line 103.
There may be some new functionality in Fibre to have "push" actions from the ODrive itself on variable changes. For now, polling will have to be used. This should probably be done in a similar way to the plot variables, by setting up another list in the store to maintain low-rate polled variables.
Part of the reason that this is complicated/messy is the python server acting as the link between ODrive USB communications and the GUI. There is work being done right now by @samuelsadok to make Fibre USB work natively in Electron, so the python server will eventually not be used. I'm not sure of the timeline for that, but it is worth mentioning. I think that the same GUI framework can be used (a list of low-rate polled parameters), but we'll just redirect the sampling calls as necessary. Thanks again for working on this!

ok, that sounds like there will be a major redesign, so for until now, i could do the polling from within vue?
I am not sure if i am capable of changing the server.py, as i am not familiar with python...

Sure, polling from within Vue is a good approach. In that case, a good pattern to use would be something similar to Axis.vue, starting at line 232.
In the created hook, a polling function is declared to call fetchParam() and then getVal(). fetchParam updates the value of the parameter in store.state.odrives, getVal actually gets that value into a variable.

Ok, i'd love to add this feature then. Two things:
  created() {
    ...
    let update = () => {
      fetchParam(this.name);
    }
    this.intervalId = setInterval(update, 1000);
    update();
  },
  beforeDestroy() {
      clearInterval(this.intervalId);
  }

Regarding polling, i'd prefer a solution like this, with clearing the interval on destroying the component. With the approach in Axis.vue, the timer is started again and again when hot-reloading during development, so after a few times, you will have many intervals running at the same time. Do you consent?
And my other question is, as far as i can see, when fetching a param, a command is sent to the server script to refresh that parameter. The only thing i haven't found yet is, how and where is the actual new value applied to the object in store and how is that then forwarded to the control that sent the request? That looks like magic at the moment to me 😅

The CtrlBoolean would look like this, accordingly. Also, the state of the checkbox is im-/exported with the dashboard:

The CtrlEnum would look like this, accordingly. Also, the state of the selectis im-/exported with the dashboard:

That looks good to me, I will have to do a more thorough review before merging. However, it looks like you are mostly done (I see some commits for each type of control)?
For fetchParam() calls - a socketio event is fired by that function in odrive_utils.js. The server receives that event, gets the new value of that parameter, and then emits a socketio event called ODriveProperty with the relevant data in a dict. On that event, a mutation is triggered in the store to update that specific parameter in store.state.odrives. So long as the Vue reactivity works correctly, it is magic 😄 .

Ok, a more thorough review. I see what you were going for with the save button idea, but to me there are two issues. One, the functionality of "flip between multiple values" is handled already by the Action components. An Action for boolean values is missing, but otherwise they already have this functionality (including saving the set value to an exported dashboard). The Controls are somewhat meant to be a transparent view/handle for the ODrive parameter.
Two: I find the duplication of set value, and getVal() value for each control to be visually messy. I'm also not sure if the save button is necessary - If the controls have live updates via the setInterval cal, and the ODrive is not disconnected, you can be quite sure that the value that you entered will make its way to the ODrive. I think that having a live update would be sufficient for this functionality (if you send a value and it "does not take", the setInterval update will set it back to what it actually is).
So to simplify things, I propose these changes: remove the usage of parentControl for the visual elements, because if the Actions are used for the "toggle value" functionality, the regular Controls do not need memory of what they were set to. Remove the duplication of a watch value and  element for the various controls, and also the save button. Keep the persistent updates on the controls, and remove "watch" elements (I think that "watch" behavior by default is OK). To maintain feature parity with the PR, could you make an ActionBoolean component?
This way, there would be a way to have saved values/toggles for all ODrive parameters via the Actions, and the Controls polled updated would function as Watch components by default. These comments are mostly to do with updated and clarifying components that already exist, and having maximum functionality per pixel on the page.
Also, thanks for fixing the Slider component and cleaning up the code in various places (especially the obvious usage of computed name in the other computed properties :smile:)! What do you think about the proposed changes?

@mdhom Any updates on the proposed changes? If not, I'd like to close this PR.


feat: added chain id to userOps

Added chainId support for the userOps

Unfortunately, the chain ID is not part of the userOp standard, so no wallet SDK or the Bundler supports it. If we introduce it to the model, it will break compatibility with the Bundler's userOp. The standard does not need it because the Ethereum transaction that wraps the userOp already contains a chain-id. github.com/blndgs/stackup-bundler/pkg/entrypoint/transaction/handleops.go#L101C84-L101C84
I see 2 workable options:

Embedding the chain-id within the Intent JSON ties the Intent to a specific network so that it's not replayable to another network if that's what we desire (I think so).
We may also accommodate multiple chain IDs, offering execution in a multi-chain environment.
-- or
Introduce it outside Intents as a single param per batch or within model.UserOperationExt
Let's raise this with the team for a decision.

Thank you for highlighting the compatibility concerns with the Chain ID integration. Based on our discussions, we've opted to embed the Chain ID within the Intent JSON, ensuring each Intent is tied to a specific network.
The change is now implemented.


Build and test the project publicly on Travis CI

This PR adds a build configuration for travis and sets the executable bits for gradlew. To actually enable CI a maintainer would need to follow these steps:

Merge this PR.
Go to https://travis-ci.org and sign in with GitHub.
Accept the Authorization of Travis CI. You’ll be redirected to GitHub.
Click the green Activate button, and select the repositories you want to use with Travis CI.

For more information see Getting Started.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: TheMrMilchmann sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

This looks great, we can turn off our private CI for auto building and now everyone can look at the actual build results. Thank you!


Recipient address rejected: User unknown in virtual mailbox table

Hi
Getting the above error when trying to send mail through the smtp server:
If I send to<EMAIL_ADDRESS>I get the error.... How do I get it to route a anything at myhost.co.za to the correct mail servers.
I can send to<EMAIL_ADDRESS>perfectly fine.
I also tried SMTP only option by SMTP stopped working all together.
docker run -d --hostname mail.myhost.co.za --name mail -p 25:25 -p 143:143 -p 587:587 -p 993:993 -v /media/andrew/storage/mailserver/maildata:/var/mail -v /media/andrew/storage/mailserver/mailstate:/var/mail-state -v /media/andrew/storage/mailserver/config:/tmp/docker-mailserver/  -e ENABLE_SPAMASSASSIN=0 -e ENABLE_CLAMAV=0 -e ENABLE_FAIL2BAN=0 -e ONE_DIR=1 -e DMS_DEBUG=1 -e PERMIT_DOCKER=network --cap-add NET_ADMIN tvial/docker-mailserver
Regards,
Andrew

Do you authenticate with user and password in the SMTP settings?

No Authentication....
Basically my needs are just and SMTP Server with DKIM setup.

you need to authenticate with smtp in the smtp settings

Hi @AndrewMoneronGenex, could post info requested in CONTRIBUTING.md?
Have you created the user?
Thanks.

Closed due to inactivity.
Reopen if needed.


[5.5] add date and cast check for dirty

Reopened PR from #18385
add tests

Your tests fail.

Hm...
Looks like i have same problem with travis as #18417
Will check in local tests.


Update Notification repo seems to be failing

Running MagicMirror in Docker image (from khassel) and I am seeing this error every 10 minutes (which aligns to the updatenotification module). I've tested this both on a raspberry pi and also on another linux box.
MM version is up to date: 2.23.0
The error I am seeing is:
[ERROR] Failed to retrieve repo info for MagicMirror: TypeError: Cannot read properties of null (reading '1')
Please let me know if there is any other information I should share.

Thanks for reporting this, I can reproduce this and will take a look within the next days

fixed with new release v2.24.0

Thanks and I validated the fix works as well.


Armor effect tweak

Tweak night vision length on mystic helmet

Any particular reason you thought this change was needed?

On a modpack I've been playing I've run into an issue where the night vision is continuously trying to apply itself but it's applying weirdly and it causes a constant flickering if wearing the helmet in the nether

Changing it to 25 fixes it? That's fairly interesting... if so I will add that change to the next version.

I haven't been able to test it due to the gradle voodoo not working for me but I noticed it's the only one that's different which makes me think it is

I am unable to replicate this bug, but I will change the application code to only apply once a second, which should fix your issue.

Also, your fix would not have worked, and would have made it worse since night vision durations under 200 ticks do the "wearing-off" flicker.

Fixed in 2.7.0.


master branch won't build, missing plugin snapshot

[error] sbt.librarymanagement.ResolveException: Error downloading org.scalanlp:sbt-breeze-expand-codegen;sbtVersion=1.0;scalaVersion=2.12:0.2.1-SNAPSHOT
[error]   Not found
[error]   Not found

known/expected, temporary? this came up in the Scala community build

Yes, sorry, I thought I fixed it, but will fix tonight
On Mon, Aug 16, 2021, 3:34 PM Seth Tisue @.***> wrote:

[error] sbt.librarymanagement.ResolveException: Error downloading org.scalanlp:sbt-breeze-expand-codegen;sbtVersion=1.0;scalaVersion=2.12:0.2.1-SNAPSHOT
[error]   Not found
[error]   Not found
known/expected, temporary? this came up in the Scala community build
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/scalanlp/breeze/issues/819, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAACLIID6WF6YTDDZAIDQGDT5GHAPANCNFSM5CIT3A2Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

(this is fixed, sorry i forgot to update)


As a user, I would like to be able to retrieve my device location instead of entering an address

Is your feature request related to a problem? Please describe.
Describe the solution you'd like
As a user, I would like to be able to retrieve my device location instead of entering an address
Additional context
Original Decidim:
Mock up
Does this issue could impact on users private data?
No, it's about performance.
Acceptance criteria

[ ]
[ ]

Decidim Core

Close per https://github.com/stadtluzern/decidim-letterbox-damagereporting/issues/11


[SUPPORT] Data loss in MOR table after clustering partition

Describe the problem you faced
As background, due to https://github.com/apache/hudi/issues/9934 we're testing out clustering our table to have fewer base files in our MOR table.
We set up a test by copying an existing table. This table only had base files (no log files) in its initial state. We wanted to verify the performance of clustering as well as data correctness. We clustered one partition and found that 261736 rows were missing after clustering.
We used the following clustering configuration (and the other configurations in "Additional Context"):
# Clustering configs
"hoodie.clustering.inline": "true",
"hoodie.clustering.inline.max.commits": 1,
"hoodie.clustering.plan.strategy.small.file.limit": 256 * 1024 * 1024,
"hoodie.clustering.plan.strategy.target.file.max.bytes": 512 * 1024 * 1024,
"hoodie.clustering.plan.strategy.sort.columns": "itemId.value",
"hoodie.clustering.plan.strategy.partition.selected": "dt=2022-08-29",
"hoodie.clustering.plan.strategy.max.num.groups": 30,

Our clustering code ran as follows:

Read one row from partition dt=2022-08-29
Write out the row (this is just a dummy way of triggering clustering inline), this update will be a no-op. We set "hoodie.clustering.plan.strategy.partition.selected" to be dt=2022-08-29 to only cluster the partition that was written to.

After the write finished I compared the clustered/unclustered tables (we had another copy before running this). Before clustering we had 399896071 rows in that partition and after clustering 399634335 rows in that partition (261736 rows were lost).
Joining the two tables, I saw that all the missing rows were from one base file that was clustered. This interestingly was the base file that received the update of 1 row:
# Spark code to find the hoodie file and record key for each of the missing rows
meta_joined_df = unclustered_df.select(
    "_hoodie_file_name",
    "_hoodie_commit_time",
    "_hoodie_commit_seqno",
    "_hoodie_record_key",
    "_hoodie_partition_path",
    "_hoodie_is_deleted",
).alias("a").join(
    clustered_df.select(
        "_hoodie_file_name",
        "_hoodie_commit_time",
        "_hoodie_commit_seqno",
        "_hoodie_record_key",
        "_hoodie_partition_path",
        "_hoodie_is_deleted",
    ).alias("b"),
    on=F.col("a._hoodie_record_key") == F.col("b._hoodie_record_key"),
    how="full_outer",
).cache()
meta_joined_df.filter(F.col("b._hoodie_record_key").isNull()).groupBy(
    F.col("a._hoodie_file_name"),
).count().alias("count").orderBy("count", ascending=False).show(
    n=10,
    truncate=False
)

Output:
+-----------------------------------------------------------------------------------+------+
|_hoodie_file_name                                                                  |count |
+-----------------------------------------------------------------------------------+------+
|f0b917f5-607e-47c4-96a4-092b4668c436-0_254-10835-21844023_20231016122622692.parquet|261736|
+-----------------------------------------------------------------------------------+------+

The deltacommit shows this file was the one that received the update:
{
  "partitionToWriteStats" : {
    "dt=2022-08-29" : [ {
      "fileId" : "f0b917f5-607e-47c4-96a4-092b4668c436-0",
      "path" : "dt=2022-08-29/.f0b917f5-607e-47c4-96a4-092b4668c436-0_20231016122622692.log.1_0-29-5280",
      "prevCommit" : "20231016122622692",
      "numWrites" : 1,
      "numDeletes" : 0,
      "numUpdateWrites" : 1,
      "numInserts" : 0,
      "totalWriteBytes" : 13402,
      "totalWriteErrors" : 0,
      "tempPath" : null,
      "partitionPath" : "dt=2022-08-29",
      "totalLogRecords" : 0,
      "totalLogFilesCompacted" : 0,
      "totalLogSizeCompacted" : 0,
      "totalUpdatedRecordsCompacted" : 0,
      "totalLogBlocks" : 0,
      "totalCorruptLogBlock" : 0,
      "totalRollbackBlocks" : 0,
      "fileSizeInBytes" : 13402,
      "minEventTime" : null,
      "maxEventTime" : null,
      "runtimeStats" : {
        "totalScanTime" : 0,
        "totalUpsertTime" : 2327,
        "totalCreateTime" : 0
      },
      "logVersion" : 1,
      "logOffset" : 0,
      "baseFile" : "f0b917f5-607e-47c4-96a4-092b4668c436-0_254-10835-21844023_20231016122622692.parquet",
      "logFiles" : [ ".f0b917f5-607e-47c4-96a4-092b4668c436-0_20231016122622692.log.1_0-29-5280" ],
      "recordsStats" : {
        "val" : null
      }
    } ]
  },
  "compacted" : false,
  "extraMetadata" : {
    "schema" : …
  },
  "operationType" : "UPSERT"
}

I inspected the replacecommit and f0b917f5-607e-47c4-96a4-092b4668c436-0 was in partitionToReplaceFileIds (360 out of 763 base files were clustered). Even stranger, the impacted base file has 523472 rows, which means exactly half the records from the base file were lost.
To Reproduce
Unclear, but I've been able to reproduce this every time on every partition I've tried.
Expected behavior
We do not expect silent data loss - I'm not sure if this is a Hudi bug or misconfiguration. This may be related to https://github.com/apache/hudi/issues/7839 which was never reproduced
Environment Description
EMR 6.10.1
Hudi version : 0.12.2-amzn-0
Spark version : 3.3.1
Hive version : 3.1.3
Hadoop version : 3.3.3
Storage (HDFS/S3/GCS..) : S3
Running on Docker? (yes/no) : Spark on Docker
Additional context
We use the following writer configurations. There were no failures in the Spark application.
'hoodie.compact.inline': True,
'hoodie.compact.inline.max.delta.commits': 6,
'hoodie.cleaner.commits.retained': 1,
'hoodie.compaction.target.io': 30000000,
'hoodie.parquet.max.file.size': 536870912,
'hoodie.parquet.block.size': 536870912,
'hoodie.parquet.small.file.limit': 268435456,
'hoodie.bloom.index.prune.by.ranges': 'false',
'hoodie.upsert.shuffle.parallelism': 9000,
'hoodie.datasource.write.payload.class': 'org.apache.hudi.common.model.DefaultHoodieRecordPayload', 'hoodie.compaction.payload.class': 'org.apache.hudi.common.model.DefaultHoodieRecordPayload',
'hoodie.rollback.parallelism': 500,
'hoodie.commits.archival.batch': 5,
"hoodie.table.name": xxx,
"hoodie.datasource.write.table.name": xxx,
"hoodie.datasource.write.operation": "upsert",
"hoodie.datasource.write.table.type": "MERGE_ON_READ",
"hoodie.datasource.write.partitionpath.field": "dt:SIMPLE",
"hoodie.datasource.write.recordkey.field": "id.value",
"hoodie.datasource.write.precombine.field": "ts",
"hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.CustomKeyGenerator",
"hoodie.datasource.write.hive_style_partitioning": "true",
"hoodie.metadata.enable": "false",
"hoodie.bootstrap.index.enable": "false",
"hoodie.index.type": "BLOOM",
"hoodie.datasource.hive_sync.enable": "true",

The table properties are:
hoodie.table.timeline.timezone=LOCAL
hoodie.table.keygenerator.class=org.apache.hudi.keygen.CustomKeyGenerator
hoodie.table.precombine.field=publishedAtUnixNano
hoodie.table.version=5
hoodie.database.name=
hoodie.datasource.write.hive_style_partitioning=true
hoodie.partition.metafile.use.base.format=false
hoodie.archivelog.folder=archived
hoodie.table.name=xxx
hoodie.compaction.payload.class=org.apache.hudi.common.model.DefaultHoodieRecordPayload
hoodie.populate.meta.fields=true
hoodie.table.type=MERGE_ON_READ
hoodie.datasource.write.partitionpath.urlencode=false
hoodie.table.base.file.format=PARQUET
hoodie.datasource.write.drop.partition.columns=false
hoodie.timeline.layout.version=1
hoodie.table.partition.fields=dt
hoodie.table.recordkey.fields=id.value
hoodie.table.checksum=3616660964

@mzheng-plaid Thanks for raising this. Couple of things we can check to triage this -

Check Spark UI and stages, if there is any stage/task failure and retry happening for same.
Try to use SIMPLE index instead of BLOOM to check if you still see the data loss. This is to triage if this issue is BLOOM index related.

@ad1happy2go no failures in Spark UI
I will check SIMPLE index instead of BLOOM

@ad1happy2go same issue with SIMPLE index

@mzheng-plaid Its a bit strange as config looks okay, I also tried to generate data and then tried compaction with your configurations but was not getting any data loss.

Can you try HoodieClusteringJob to cluster the data. - https://hudi.apache.org/docs/next/clustering#hoodieclusteringjob
Can you try OSS 0.12.3 version once.
Can you try reproduce this issue with some dummy data. That can help us resolve this issue.

@ad1happy2go sure I can try [3] and if I can't I'll try [1] and [2] - could you share your test data, test code, and testing configs? I'd like to start there and see if I can see any differences in our setups.

@ad1happy2go  I haven't been able to reproduce this with dummy data at ~50gb scale
from pyspark.sql.types import StringType
from pyspark.sql import functions as F
import uuid
import math

# Size calculation
# Each UUID is 36 characters, and we can estimate 2 bytes per character (UTF-8 encoding)
# Adding some overhead for Row object storage, let’s use 100 bytes as a rough estimate per row
row_size_bytes = 100

# Target size for the DataFrame in bytes (50gb)
target_size_bytes = 50 * 1024 * 1024 * 1024

# Calculate the number of rows needed to generate the DataFrame
num_rows = math.ceil(target_size_bytes / row_size_bytes)
print(num_rows)

uuid_udf = F.udf(lambda: str(uuid.uuid4()), StringType())
df = spark.range(num_rows).withColumn("id", uuid_udf())
df = df.withColumn("partition", F.lit("a"))
df.show(5, truncate=False)

536870912
+------------------------------------+---------+
|id                                  |partition|
+------------------------------------+---------+
|fec3f590-f3f5-41bb-8c8c-0542ebf9e911|a        |
|383880f2-943c-40f0-abe1-617a026b6848|a        |
|4252b669-8d57-4346-8e1f-58bf75a7e553|a        |
|d0152ad5-83e7-4e94-a488-3f09b28335ce|a        |
|205f1853-61ee-4f5c-9388-b39cca5a1a67|a        |
+------------------------------------+---------+
only showing top 5 rows

I upserted all of these rows, then clustered with a dummy upsert. I'm not sure how to reproduce this issue with dummy data - this seems to be only a problem at a large scale
The biggest clues I have are:

partition for dataset where this failed is ~150gb and 400 million rows
the base file touched by the upsert is the only one affected. If I upsert partition X but cluster partition Y, there is no data loss.
exactly half the rows in the base file are missing after clustering (are base files internally split in some way? some sort of grouping?)
clustering replaced the base file that was upserted to

I'm not sure at what point I'll have more bandwidth to dig into this issue more, but given the seriousness of silent data loss we're just abandoning using clustering at this point.

@mzheng-plaid Thanks for the update. I really appreciate it.

@ad1happy2go @codope I was able to reproduce with the following Spark code (50000 row dataset). It seems the problem is related to handling of array fields in structs. Could you confirm if you're able to reproduce using this code?
from pyspark.sql.types import StringType
from pyspark.sql import functions as F
from pyspark.sql import types as T

import uuid
from pyspark.sql import Row
import random

hudi_options = {
    "hoodie.table.name": "clustering_bug_test",
    "hoodie.datasource.write.recordkey.field": "id.value",
    "hoodie.datasource.write.partitionpath.field": "partition:SIMPLE",
    "hoodie.datasource.write.table.name": "clustering_bug_test",
    "hoodie.datasource.write.table.type": "MERGE_ON_READ",
    "hoodie.datasource.write.operation": "upsert",
    "hoodie.datasource.write.precombine.field": "publishedAtUnixNano",
    "hoodie.datasource.write.payload.class": "org.apache.hudi.common.model.DefaultHoodieRecordPayload",
    "hoodie.compaction.payload.class": "org.apache.hudi.common.model.DefaultHoodieRecordPayload",
    # Turn off small file optimizations
    "hoodie.parquet.small.file.limit": "0",
    # Turn off metadata table
    "hoodie.metadata.enable": "false",
    "hoodie.datasource.write.keygenerator.class": "org.apache.hudi.keygen.CustomKeyGenerator",
    # Hive style partitioning
    "hoodie.datasource.write.hive_style_partitioning": "true",
    'hoodie.cleaner.commits.retained': 1,
    "hoodie.bootstrap.index.enable": "false",
    'hoodie.commits.archival.batch': 5,
    # Bloom filter
    "hoodie.index.type": "BLOOM",
    'hoodie.bloom.index.prune.by.ranges': 'false',
}
clustering_hudi_options = {
    **hudi_options,
    "hoodie.clustering.inline": "true",
    "hoodie.clustering.inline.max.commits": 1,
    "hoodie.clustering.plan.strategy.small.file.limit": 256 * 1024 * 1024,
    "hoodie.clustering.plan.strategy.target.file.max.bytes": 512 * 1024 * 1024,
    "hoodie.clustering.plan.strategy.sort.columns": "id.value",
    "hoodie.clustering.plan.strategy.max.num.groups": 300,
}

random.seed(10)
dummy_data = [
    Row(
        id=Row(value=str(uuid.uuid4())),
        publishedAtUnixNano=i,
        partition="1",
        struct_array_column=Row(
            element=[str(random.randint(0, 1000000)) for i in range(random.randint(1, 100))],
        ),
        struct_column=Row(
            nested_array_column=Row(
                element=[str(random.randint(0, 1000000)) for i in range(random.randint(1, 100))],
            ),
        ),
        # This padding ensures files are large enough to reproduce the data loss
        **{
            f"col_{i}": str(uuid.uuid4())
            for i in range(100)
        },
    )
    for i in range(50000)
]
df_dummy = spark.createDataFrame(dummy_data)

# This was tested in S3
PATH = f"{OUTPUT_PATH}"
df_dummy.write.format("hudi").options(**hudi_options).mode("append").save(PATH)

read_df = spark.read.format("hudi").load(PATH)
data = read_df.take(1)
init_count = read_df.count()

# This upsert should be a no-op (re-writing 1 existing row)
upsert_df = spark.createDataFrame(data, read_df.schema)
upsert_df.write.format("hudi").options(**clustering_hudi_options).mode("append").save(PATH)

read_df = spark.read.format("hudi").load(PATH)
final_count = read_df.count()
print(f"{init_count}, {final_count}")

The schema is:
root
 |-- id: struct (nullable = true)
 |    |-- value: string (nullable = true)
 |-- publishedAtUnixNano: long (nullable = true)
 |-- partition: string (nullable = true)
 |-- struct_array_column: struct (nullable = true)
 |    |-- element: array (nullable = true)
 |    |    |-- element: string (containsNull = true)
 |-- struct_column: struct (nullable = true)
 |    |-- nested_array_column: struct (nullable = true)
 |    |    |-- element: array (nullable = true)
 |    |    |    |-- element: string (containsNull = true)
 |-- col_0: string (nullable = true)
 |-- col_1: string (nullable = true)
 |-- col_2: string (nullable = true)
 |-- col_3: string (nullable = true)
...

We expect init_count and final_count to be the same but it's actually (may vary)
50000, 48000

@mzheng-plaid I looked into your code and found out that number of fields in your dataset is more than 100 which is the cause for this issue. I set this config ("spark.sql.codegen.maxFields","120")  and it worked for me. After setting this there is no duplicates.
Can you please try with this config, and if your original dataset also have more than 100 columns then this config may work for you. Thanks.

Thanks @ad1happy2go , that is extremely surprising to me - could you explain why there is a dependency on spark.sql.codegen.maxFields?
I will try this out.

I just found reference to https://github.com/apache/hudi/issues/8416 and https://github.com/apache/hudi/pull/7334 - is this the issue?
This seems extremely bad but it says the fix was to be in 0.12.1? How come this wasn't backported to 0.12.x?

Yes, They may be related. We missed to back port to 0.12.X minor releases. Does your original dataset also have more than 100 columns?

Yes it does, I will validate on the full dataset but this seems to match exactly the symptoms we saw originally.
I'm not following how https://github.com/apache/hudi/pull/7334 relates to the fix (ie. why this is related to spark.sql.codegen.maxFields needing to be expanded). Could you explain this link?
Does this affect other table services and/or CRUD operations on MOR tables? Thanks

Sorry for the late response here @mzheng-plaid . This is spark config which controls the total number of fields in a schema that is acceptable for whole-stage code generation.
Did after adding this config/patch worked for you?

hey @ad1happy2go : whats the follow up on this. do we need to make any fixes to hudi. or doc enhancements etc?


Add support for inner spaces within parentheses and square brackets.

First of all, thank you for this project!
Also fixed one failing test before adding my changes (commits are logically split).
See the test files for the expected outcomes of new formatting options.

First of all, there is no need to divide into two options, and pairs of space option must be selected at the same time. and then you should consider use Add<KeepElement>(1) instead of Add<SpaceElement>()

Very good suggestion. I merged the pairs of options and gave them more reasonable names.

local t = {}
function t:fffff(
aaa
)
    local t = 13
end

format this code multiple times and the result is unstable

t:fffff( 1, 2, 3 )

codestyle check warning:   "unnecessary whitespace after operator '('"

my formatter can check code style. you can use vscode emmylua-codestyle plugin test this feature

t:fffff( 1, 2, 3 )

codestyle check warning: "unnecessary whitespace after operator '('"

Yeah that's because it's a different style. I think the user should be able to choose what style they prefer.

local t = {}
function t:fffff(
aaa
)
    local t = 13
end

format this code multiple times and the result is unstable

I tested this with the default settings by running:
CodeFormat format -f test.lua

and with new options enabled:
CodeFormat format -f test.lua -c lua.template.editorconfig

The results are the same if I reformat multiple times. Are you sure it's not VsCode messing it up?

Is there a way to configure VSCode to use this build for your extension for testing?

Good catch, I was able to reproduce it. I'll try to fix it.

clone https://github.com/CppCXY/VSCode-EmmyLuaCodeStyle for vscode extension
do
yarn  #or npm install

press F5, the vscode extension will try to connect localhost's 5008 port.
you can run with CodeFormatServer 5008  debug with gdb or visualstudio or clangd

I've fixed the parentheses options. I still need to verify the square brackets option and do some more testing.

Thanks to your instructions, I was able to run the extension in vscode with my local server.
I could now see the warnings you mentioned.
All fixed now (fingers crossed :wink:).


Convert the protocol class into an enum abstract over an int

Closes #44.
Works around any name clashes with ERROR by converting the class into an enum abstract so the compiler will inline the values. Added an implicit to Int so this shouldn't be a breaking change for anyones code.

Thank you @Aidan63 🎉 🎉 don't hesitate to shout out if you happen to find any other issues


Change order that PostProcess Processors are run (#6445)

Make sure Processors that work on full links are run first so that
something matching another pattern doesn't alter a link before we get to
it, for example:
https://stackoverflow.com/questions/2896191/what-is-go-used-fore
Fixes #4813
This is backport to release/v1.8

@mrsdizzie thanks for quick backport 😄

My Pleasure :)


Consider adding test for TypeScript server plugins

In vscode 1.64.0, all extension functionality that used TypeScript server plugins did not work at all on Windows #142139 (ex. the Deno extension, Styled Components, etc.). The way this error occurred indicates that there's no tests for TypeScript server plugins. Perhaps it would be useful to add a basic test for this functionality to help reduce the chance of this happening in the future.

I doubt the benefits of a test outweigh the cost of maintaining a test that spans three different products: vscode, TS server, and a plugin. This is first time we've seen a failure of this nature
To help, make sure you are testing your extension on VS Code insiders and share feedback if something no longer works as expected

@mjbvz couldn't this just be a test in typescript-language-features? I don't think it would span three products, but I don't understand the vscode codebase that well.
Using insiders doesn't seem realistic because something could break that was merged between the insiders build someone downloaded and the new release.

Insiders tracks the next stable version of VS Code that will ship, besides a few minor differences like product name and icon
In this case, TS plugins on windows were broken in insiders for around 3 weeks. We would have been able to fix this issue quickly if we had seen reports of the issue during that period. You can help us by testing your extension on insiders and letting us know if something regresses