mlforecast: can the number of exogenous variables be different for different unique_id?

Description
Hello, I want to use nixtla mlforecast for prediction. My question is if the number of exogenous variables could be different for different unique_id? My training data may look like this: For BE and CB, the number of their exogenous variables is different.
unique_id	ds	y	Exogenous1	Exogenous2	day_0	day_1	day_2	day_3	day_4	day_5	day_6
0	BE	2016-12-01 00:00:00	72.00	61507.0	71066.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0
1	BE	2016-12-01 01:00:00	65.80	59528.0	67311.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0
2	BE	2016-12-01 02:00:00	59.99	58812.0	67470.0	0.0	0.0	0.0	1.0	0.0	0.0	0.0
3	CB	2016-12-01 03:00:00	50.69	57676.0
4	CB	2016-12-01 04:00:00	52.58	56804.0

Thanks for your attention.
Use case
No response

Hey @ahbon123, thanks for using mlforecast. The models are trained using all of the series, so it depends on what you want to do, you could for example fill the missing features with NaNs and let the model handle them or split them and model them separately.


CIP 0105 CLI support

This PR builds on top of the previous one and adds support of CIP 0105 to CLI. In particular the following was added:

Support of derivation of all varieties of drep, cc_cold and cc_hot keys, namely extended signing keys, extended verification key and verification keys.

cardano-address key child 1852H/1815H/0H/5/0 < root.xsk > hot.xsk
cardano-address key public --with-chain-code < hot.xsk > hot.xvk
cardano-address key public --without-chain-code < hot.xsk > hot.vk

Support of making hash from verification keys (both varieties):

cardano-address key hash < hot.xvk > hot
cardano-address key hash < hot.vk > hot

Support for using drep, cc_coldandcc_hot` keys, namely extended verification key and verification keys, and verification key hashes in scripts and producing validation, preimage and hash of the script.

cardano-address script validate "all [$(cat drep),active_from 5001]"
cardano-address script preimage "all [$(cat drep),active_from 5001]"
cardano-address script hash "all [$(cat drep),active_from 5001]"

All produced keys assume bech32 prefixes as specified in CIP-0105.

@angerman ci/* seems to be RED - do you know how to make it work?


Release v0.29.2- Duplicate records when searching

hello everybody,
When searching, the record is duplicated, also when searching, the relationships not charged correctly! in my case i have Model: "PROPERTY" hasMany(image) and belongsTo(user). The relationship "image" is empty for last record indexed
https://user-images.githubusercontent.com/45343853/201807917-dd5bcf38-d7f2-44ae-a58f-cad0ceea5113.mp4
Meilisearch version: [e.g. v0.29.1]
Additional information that may be relevant to the issue.
[e.g. Windows 11 Professionnel, Chrome]

Hello @seifrached
I think it's related to this: https://github.com/meilisearch/meilisearch/issues/3021
@meilisearch/core-team can you confirm this is a duplicate issue?
We will release v0.29.2 today that should fix this problem 😊
Sorry for the inconvenience

@curquiza
also with v0.29.2 the bug persists

Thanks for the report @seifrached we'll investigate this soon

Hey @seifrached,
Unfortunately, the bug in v0.29.1 corrupted your database, the only way you can recover your data is to send again your documents in a fresh index.
The bug that was part of v0.29.0 and v0.29.1 was considering freshly deleted documents as not deleted and reinserting them in the database. This is the reason why you can't even do a dump export.
I am sorry if it wasn't clear. We will change the changelog to help users recover from that and reduce confusion about that bug.

Hello @Kerollmops
I hope that the problem will be solved, and for my part, if there is something new, I will tell you 👍

Hey @seifrached,
The problem should be fixed by now on v0.29.2. You must reimport your dataset on a fresh install and not run your Meilisearch on an existing database. You can't use a dump to import your data. You must do HTTP calls to send your dataset and settings again. Have you tried that to check that the bug is fixed?

Hey @seifrached,
The problem should be fixed by now on v0.29.2. You must reimport your dataset on a fresh install and not run your Meilisearch on an existing database. You can't use a dump to import your data. You must do HTTP calls to send your dataset and settings again. Have you tried that to check that the bug is fixed?

It is currently working, "no duplicate records", but after checking i found that the problem lies in the settings !
When setting up rankingRules or synonyms or typo Tolerance the relationships does not retrieved.

Hum... looks like it is another issue, then.
Could you please clarify the relationships does not retrieved sentence? I am not sure to catch the meaning here.
Would it be possible to open a new bug report issue in this repository with a minimal reproducible example?


Filter use by percentage instead of hours?

Would it be possible to add an attribute that indicates the percentage of filter_life_remaining in addition to the filter_hour currently available?

All the data is obtained using this Winix library. Unfortunately we only have access to filter_hour and filter_replace_date.


Add API spec in the README

Fixes the first item in #77
@noahdietz @elsapeng

@mohsen1, updated :)

Also, right now the first 3 config items are required, we can change to optional and set default value (we did that in swagger-node, but not in this module) if you think this make more sense @mohsen1 @noahdietz

No, no need to change the program because of the docs. If the load test stuff is not in master yet, let's hold on to it and add it later. Other than that it looks good. Thanks for fixing it.

thanks for changing this @elsapeng! looks good

updated :)

:+1:

thank you!


Set Azure ems subscription to default subscription if not already set.

At the moment we do not require a subscription ID for an Azure cloud provider, but just recently we added the ability to specify a subscription ID, and added a backing column in the database.
However, the refresher doesn't currently use it by default. It needs to use the specified subscription ID if provided, or it needs to set and save the default subscription ID if one is not provided. This does both.

@blomquisg @bronaghs Please review.

LGTM :+1:

Seems fine to me.

WOOT!


Clublife

Clublife is one of the static pages.

@97yogitha Add only the files you've changed. Do not use git add .. Don't commit .pyc files and log files. Only add those files that you've changed to the commit.

@abhijithanilkumar how do i make the changes now?

@rohitvarkey I am unable to display images if there are saved in a local folder. Let me know what need to be done.

Check this link. And images has to be stored in the folder mentioned as MEDIA_ROOT in the settings.py file.

@97yogitha Or you can just remove the unwanted files and commit again. Like this :stuck_out_tongue:

@97yogitha As @abhijithanilkumar mentioned, the image file needs to go here.
The Django docs will help you understand how to access it then https://docs.djangoproject.com/en/1.9/howto/static-files/

screenshot

This is for the hobby clubs need to add more cards..

@jaidevanoop I added jumbotron look into it. But there is an issue i cant center the  buttons. Let me know what needs to be done.

@97yogitha Delete local branch, do a git pull and you'll be set. Get that tech club thing enabled by default and this will be ready for a merge!
Good work! :+1:!

@rohitvarkey Changed the tech block to default

@97yogitha Merged! :smile:


Ionic: Uncaught (in Promise)....probably relating to NgZone.runOutsideAngular

Version info

Angular: 5.2.11
Firebase: latest
AngularFire: latest
Ionic: latest
How to reproduce these conditions
Steps to set up and reproduce
Setup a new project and initialize the Firebase app, import AngularFireStore
and setup a simple homepage containing only:
constructor(public store:AngularFirestore , public navCtrl: NavController, public navParams: NavParams) {
  }

and my "store" is not even called yet
Sample data and security rules
No data in Database yet and security rules are in test mode
** Screenshots **
https://snag.gy/9zfH3T.jpg
Expected behavior
The Page loads normally
Actual behavior
Showing Error message (see Screenshot via link above)

I also have the same issue, anyone can help here?

Hey there! I found a solution for that.
npm i<EMAIL_ADDRESS>rxjs-compat --save
and then the problem was gone :-)

This also solved it for me. Was getting this error while following along with a tutorial, thank you a lot there @obermoserkitz.


Error: Mixing symbol graph files with and without extension declarations is not supported

Description
When building the documentation for FluentKit using Xcode 15b7, I get this error. I do not get the error with Xcode 14.
CompileDocumentation /Users/rmann/Downloads/fluent-kit-main/Sources/FluentKit/Docs.docc (in target 'FluentKit' from project 'fluent-kit')
    cd /Users/rmann/Downloads/fluent-kit-main
    /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/docc convert --index --fallback-display-name FluentKit --fallback-bundle-identifier FluentKit --fallback-bundle-version 0 --output-dir /Users/rmann/Library/Developer/Xcode/DerivedData/fluent-kit-main-gaueyemdnqujlcggopsdaslfbnfg/Build/Products/Debug-iphoneos/FluentKit.doccarchive --ide-console-output --diagnostics-file /Users/rmann/Library/Developer/Xcode/DerivedData/fluent-kit-main-gaueyemdnqujlcggopsdaslfbnfg/Build/Intermediates.noindex/fluent-kit.build/Debug-iphoneos/FluentKit.build/FluentKit-diagnostics.json /Users/rmann/Downloads/fluent-kit-main/Sources/FluentKit/Docs.docc --additional-symbol-graph-dir /Users/rmann/Library/Developer/Xcode/DerivedData/fluent-kit-main-gaueyemdnqujlcggopsdaslfbnfg/Build/Intermediates.noindex/fluent-kit.build/Debug-iphoneos/FluentKit.build/symbol-graph

Error: Mixing symbol graph files with and without extension declarations is not supported.

Symbol graph files with extension declarations:
<EMAIL_ADDRESS><EMAIL_ADDRESS>Symbol graph files without extension declarations:
<EMAIL_ADDRESS>
Checklist

[ ] If possible, I've reproduced the issue using the main branch of this package.
[X] This issue hasn't been addressed in an existing GitHub issue.

Expected Behavior
I expect it to build. I don't know how to control this, or even what it’s really complaining about.
Actual behavior
No response
Steps To Reproduce
Build docs for FluentKit using Xcode 15b7
Swift-DocC Version Information
No response
Swift Compiler Version Information
% swiftc --version
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-<IP_ADDRESS>.106 clang-15<IP_ADDRESS>)
Target: arm64-apple-macosx13.0

This was fixed by #715 (and #725 for the 5.10 release)


Add conversions from and into Vec3 for Color

Objective
Add conversions from and into Vec3 for Color.
Changelog

Implemented From<Vec3> for Color.
Implemented From<Color> for Vec3.
Implemented From<Color> for [f32; 3].

Migration Guide
No migration guide

I don't think we should do this. Color math is really easy to screw up in subtle ways, and this makes the conversion more implicit.

No problem! I felt like Vec4::from(color).truncate() is a bit boilerplate-y, so I opened this PR.

Ah, the existing impls really help motivate this PR. Lemme open an issue to discuss removing them.


New tools, books and blogs

Added Shikra under Hardware Tools. Created new Bluetooth/BLE, ZigBee, SDR, and RFID/NFC tools subsections. Added 5 books. Added 6 blogs.

Still seems to be a merge conflict in the Books section @jarv-git . Otherwise, LGTM now and linter does not complain either. What do you think @weidenba

I resolved the merge conflict and converted the technology based tools to subsections. I think we can merge it now. The linter complains about apimote.com link being dead. However, I checked it manually and it seems to work.

Merged and thanks for the contribution @jarv-git !


tracker(dm): schema tracker without embedded TiDB

What problem does this PR solve?

Issue Number: close https://github.com/pingcap/tiflow/issues/5334
What is changed and how it works?
Exec now accepts ast.StmtNode, and SchemaTracker should do the preprocessing works like TiDB executor.
I hope other API is not effected
Check List
Tests

pass old tests

Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
DM no longer uses embedded TiDB to track table structure from upstream DDL

/run-dm-integration-test

/run-dm-integration-test

now new SchemaTracker will not report error like "unsupported add column 'c' constraint UNIQUE KEY", many tests need to be updated :joy:
/run-dm-integration-test

/run-dm-integration-test

/run-dm-integration-test

/run-dm-integration-test

/run-dm-integration-test

https://ci2.pingcap.net/blue/organizations/jenkins/dm_ghpr_integration_test/detail/dm_ghpr_integration_test/8816/pipeline
DM integration test passed. Most changes are test files, ptal @GMHDBJD @D3Hunter @lichunzhu

Codecov Report

Merging #6659 (4a6381b) into master (2ea67ba) will increase coverage by 0.1173%.
The diff coverage is 72.5766%.

Flag
Coverage Δ

cdc
66.0099% <74.7422%> (+0.1418%)
:arrow_up:

dm
51.9530% <58.1699%> (-0.0887%)
:arrow_down:

engine
62.8505% <73.3238%> (+0.3144%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.
@@               Coverage Diff                @@
##             master      #6659        +/-   ##
================================================
+ Coverage   59.2138%   59.3312%   +0.1173%
================================================
  Files           780        785         +5
  Lines         88405      89174       +769
================================================
+ Hits          52348      52908       +560
- Misses        31415      31610       +195
- Partials       4642       4656        +14

/cc @GMHDBJD @lichunzhu

PTAL @GMHDBJD @lichunzhu

/merge

/run-dm-integration-test
/run-integration-test
/run-dm-compatibility-test

/run-integration-test
/run-dm-compatibility-test

/run-dm-integration-test
/run-integration-test

/run-all-tests

/run-verify


Trigger chart release when policy-server is released

Updates the release CI/CD pipeline to trigger the kubewarden-controller and kubewarden-crds charts updates.
We might use this action to dispatch workflows

Moving this to block until we fix this issue


Interface Overview arrows point to wrong side if Sidebar has been moved to the right.

VSCode Version: 1.24.1 (24f62626b222e9a8313213fb64b10d741a326288)
OS Version: macOS 10.13.4

Steps to Reproduce:

cmd-P Toggle Side Bar Position (so it's on the right)
cmd-P User Interface Overview

Does this issue occur when all extensions are disabled?: Yes

I imagine the overlay is a pre-generated image.

/duplicate https://github.com/Microsoft/vscode/issues/18861


Bug-fixes on boot() method based on newer dependencies

Fixes Symfony\Component\Debug\Exception\FatalThrowableError: Call to undefined method Illuminate\Foundation\Application::share() in .../vendor/sven/flex-env/src/FlexEnvServiceProvider.php:16 Error

Thanks! However, I'd prefer to do it like I did in Artisan View. Just pushed a commit to do that 😃

Even better! Thanks!


Fix error in test

The error was only in the deployment of the tests, because of the list of nodeIDs not being necessarily sorted when retrieved from the map of weights. This paired with the uneven weights meant that in the end we sometimes crashed a replica that took enough power to prevent delivery of any transaction.
A simple word fixes the bug.

Upon further testing 30 seconds is not enough for the test, but 40 is. So let us leave it at that.


orgs filtering fix in patients list

fixes per conversation with Justin and address issues found related to this story: https://jira.movember.com/browse/TN-124

remove select filter for organization column
preserve filters based on selection(s) from filter list control

@OptimusRhine Adam, it would be nice if you can have a look and see if my changes in patients.py are okay?  I figure it is easier to get filtered rows for selected organization(s) from the server side rather than client side

Hello @achen2401! Thanks for submitting the PR.

In the file portal/views/patients.py, following are the PEP8 issues :

Line 58:40: E128 continuation line under-indented for visual indent


The remote procedure call failed

Windows Version
10.0.22623.1037
WSL Version
<IP_ADDRESS>
Are you using WSL 1 or WSL 2?

[X] WSL 2
[ ] WSL 1

Kernel Version
<IP_ADDRESS>
Distro Version
Ubuntu-18.04
Other Software
No response
Repro Steps
Open WSL
Expected Behavior
Run correctly
Actual Behavior
Say "The remote procedure call failed"
Diagnostic Logs
No response

me too

same here

1.1.5 WSL from MicrosoftStore
Windows version: 22H2, 19045.2728

when I launch the wsl, it return "The remote procedure call failed"

PS C:\Users\ingre> wsl -v
WSL 版本： <IP_ADDRESS>
内核版本： <IP_ADDRESS>
WSLg 版本： 1.0.50
MSRDC 版本： 1.2.3770
Direct3D 版本： 1.608.2-61064218
DXCore 版本： 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows版本： 10.0.22621.1413
PS C:\Users\ingre> wsl
远程过程调用失败。
Error code: Wsl/Service/RPC_S_CALL_FAILED
PS C:\Users\ingre>

This happened after updating wsl yesterday, how to solve it?

This happened after updating wsl yesterday, how to solve it?

Reinstall version 1.1.3 temporarily solve this problem for me

This happened after updating wsl yesterday, how to solve it?

Reinstall version 1.1.3 temporarily solve this problem for me
It's solved, thank you

This happened after updating wsl yesterday, how to solve it?

Reinstall version 1.1.3 temporarily solve this problem for me
It's solved, thank you

If somebody is in this state, could you please collect a dump of wslservice.exe? I'm not seeing this crash locally, might be something specific to your locale.
(As admin)

sc stop wslservice
sc start wslservice
windbg -p
in the debugger, "g" to continue.
run wsl command
Observe crash
.dump /ma <path_to_dump>
zip and upload dump

Hi!
I reproduced it in my environment, so I share the information below.
It is a ja_JP environment.

https://drive.google.com/file/d/1R3dbDfOsm2Zbk_KngCu3V0tKodj3-K2M/view?usp=sharing

@mirror-kt - thanks for the dump. I've verified this is an issue we fixed and will be releasing in 1.1.6 soon. In the meantime I would suggest clearing the contents of your %userprofile\.wslconfig file, because there are some invalid values in there (namely "localhostForwarding").

I've removed the link to your dump since I have what I need. Thanks!

After locating the .wslconfig file in C:\Users<username>, and after removing the settings below, it returned to normal.
[system-distro-env]
WESTON_RDP_MONITOR_REFRESH_RATE=144

After locating the .wslconfig file in C:\Users<username>, and after removing the settings below, it returned to normal.
[system-distro-env]
WESTON_RDP_MONITOR_REFRESH_RATE=144

在 中找到文件并删除以下设置后，它恢复正常。.wslconfig``C:\Users<username>
[system-distro-env]
WESTON_RDP_MONITOR_REFRESH_RATE=144

But there is no such content in my .wslconfig file

@EricFengFuuuu
Have you tried deleting unnecessary settings?

[network]
generateResolvConf = false

in my case, delete this part from my config works, really weird

[user]
default=stark

I delete this section solve my problem.
After taht, I try to put it back, the WSL will failed to start in several seconds.

I had the same problem, open the ".wslconfig", then opening the WSL once after commenting (deleting) the "swapFile", then recovering the entry and opening the WSL again.

me too，thanks!

delete "swapFile" is work for me
me too,thanks!

empty ".wslconfig" worked for me

Fixed with https://github.com/microsoft/WSL/releases/tag/1.1.6.

this is still happening to me, some how


Error: No such file or directory: python2.7/site-packages/altanalyze/Config//arrays.txt

I just installed AltAnalyze through pip in conda environment (with python 2.7) and following instructions here tried to install the organism database with
altanalyze --species Dr --update Official  --additional all --version EnsMart102
This threw the following error (I've replaced some paths with /path/to/):
Current database version: EnsMart102
AltAnalyze.py --species Dr --update Official --additional all --version EnsMart102
Traceback (most recent call last):
  File "AltAnalyze.py", line 8435, in runCommandLineVersion
    commandLineRun()
  File "AltAnalyze.py", line 7951, in commandLineRun
    array_codes = UI.remoteArrayInfo()
  File "/path/to/lib/python2.7/site-packages/altanalyze/UI.py", line 3679, in remoteArrayInfo
    importArrayInfo()
  File "/path/to/lib/python2.7/site-packages/altanalyze/UI.py", line 3685, in importArrayInfo
    for line in open(fn,'rU').readlines():
IOError: [Errno 2] No such file or directory: '/path/to/lib/python2.7/site-packages/altanalyze/Config//arrays.txt'

I don't know what should I do with it. Any help is appreciated.

Before the above, with --version EnsMart102, I tried without the --version : altanalyze_ICGS2]$ altanalyze --species Dr --update Official  --additional all and got this error:
AltAnalyze depedency not met for: wx
...The AltAnalyze Results Viewer requires wx

WARNING!!!! Some dependencies are not currently met.
This may impact AltAnalyze's performance

Running commandline options

Arguments input: ['AltAnalyze.py', '--species', 'Dr', '--update', 'Official', '--additional', 'all']

AltAnalyze.py --species Dr --update Official --additional all
Downloading the following file: species_all.txt   Connection opened. Downloading (be patient)
*

Downloading the following file: source_data.txt   Connection opened. Downloading (be patient)
*

Downloading the following file: array_versions.txt   Connection opened. Downloading (be patient)
* *

Finished downloading the latest configuration files.
Species name to update: Danio rerio
Ensembl version current
Traceback (most recent call last):
  File "AltAnalyze.py", line 8435, in runCommandLineVersion
    commandLineRun()
  File "AltAnalyze.py", line 7975, in commandLineRun
    for ad in db_versions_vendors[db_version]:
KeyError: 'current'

EnsMart102 is not currently provided unless you end up building it on your
end which is possible. Can you run with --version EnsMart72 ? This is an
older version of Ensembl but the default supported version (EnsMart100 is
build an in testing but not yet uploaded).
If required, let me know.
Best,
Nathan
On Mon, Dec 14, 2020 at 2:52 PM therealgenna<EMAIL_ADDRESS>wrote:

Before the above, with --version EnsMart102, I tried without the --version
: altanalyze_ICGS2]$ altanalyze --species Dr --update Official
--additional all and got this error:
AltAnalyze depedency not met for: wx
...The AltAnalyze Results Viewer requires wx
WARNING!!!! Some dependencies are not currently met.
This may impact AltAnalyze's performance
Running commandline options
Arguments input: ['AltAnalyze.py', '--species', 'Dr', '--update', 'Official', '--additional', 'all']
AltAnalyze.py --species Dr --update Official --additional all
Downloading the following file: species_all.txt   Connection opened. Downloading (be patient)
*
Downloading the following file: source_data.txt   Connection opened. Downloading (be patient)
*
Downloading the following file: array_versions.txt   Connection opened. Downloading (be patient)

Finished downloading the latest configuration files.
Species name to update: Danio rerio
Ensembl version current
Traceback (most recent call last):
File "AltAnalyze.py", line 8435, in runCommandLineVersion
commandLineRun()
File "AltAnalyze.py", line 7975, in commandLineRun
for ad in db_versions_vendors[db_version]:
KeyError: 'current'
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/nsalomonis/altanalyze/issues/48#issuecomment-744672007,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABD5TSKBWJL4QHBFERTYY6LSUZUJLANCNFSM4U3HKBOQ
.

Now I see that arrays.txt sits in a different place, under my home directory ~/altanalyze/Config/, and this is not where the script is looking for it -  in /path/to/lib/python2.7/site-packages/altanalyze/Config//arrays.txt which is under the python installation! Under pythin installation, in the Config dir I have the following files:
array_versions.txt  report.log  source_data.txt  species_all.txt  version.txt

Hi Nathan, thank you for a quick response! No, just tried - it's the same with --version EnsMart72. I think it's because different files got into different places, as I posted above. Where should the files like array.txt be? It seems strange that the program is looking for it under the python directory tree ...

The files should be OK in either place, but it’s possible we are missing
something. As a sanity check, can you download the software from GitHub
from
https://github.com/nsalomonis/altanalyze
and try again?
On Mon, Dec 14, 2020 at 4:42 PM therealgenna<EMAIL_ADDRESS>wrote:

Hi Nathan, thank you for a quick response! No, just tried - it's the same
with --version EnsMart72. I think it's because different files got into
different places, as I posted above. Where should the files like array.txt
be? It seems strange that the program is looking for it under the python
directory tree ...
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/nsalomonis/altanalyze/issues/48#issuecomment-744728479,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABD5TSMAJSU7N4DMSCQVGVDSU2BCVANCNFSM4U3HKBOQ
.

I downloaded, unzipped and ran python AltAnalyze.py --species Dr --update Official  --additional all --version EnsMart72 which worked. Is EnsMart72 the latest I can use? I don't want to burden you with creating a newer version if that's the latest - I am not sure I'll need it. I just want to run ICGS2 on scRNAseq to do clustering.
So would you recommend me to just stick with the downloaded version and ignore the one with pip install altanalyze?

It shouldn’t honestly matter too much (better to have the latest GitHub
version honestly), since it will primarily impact default pathway analyses.
Are you combining multiple h5, mtx or text files? If so, there is a simple
option to do this by adding all of the input files to one folder and
setting that as the target path for ChromiumSparseMatrix
Having R and Biobase and hopach in R will improve the clustering.
Best,
Nathan
On Mon, Dec 14, 2020 at 5:12 PM therealgenna<EMAIL_ADDRESS>wrote:

I downloaded, unzipped and ran python AltAnalyze.py --species Dr --update
Official --additional all --version EnsMart72 which worked. Is EnsMart72
the latest I can use? I don't want to burden you with creating a newer
version if that's the latest - I am not sure I'll need it. I just want to
run ICGS2 on scRNAseq to do clustering.
So would you recommend me to just stick with the downloaded version and
ignore the one with pip install altanalyze?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/nsalomonis/altanalyze/issues/48#issuecomment-744742724,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABD5TSNCVEIUNA6Z3FV5P7LSU2ETDANCNFSM4U3HKBOQ
.

Great, thank you so much! I guess I'll have to extract count matrices and make text files from Seurat objects which were made/merged from multiple 10X run outputs.

FYI, I noticed that with pip install I get Config.zip and AltDatabase.zip in the /path/topython2.7/site-packages/altanalyze but there are no unzipped versions. When I try to install the database as above, the ~/altanalyze is created in my home directory, under which I also get ~/altanalyze/Config and ~/altanalyze/AltDatabase, but then I get the reported error.
Moving Config and AltDatabase from ~/altanalyze to /path/topython2.7/site-packages/altanalyze works (and I guess unzipping the zip files would work as well).

That is fine or you can add all Cell Ranger sparse matrix outputs to one folder with the appropriate sample names added (e.g., cancer_rep1.h5). ICGS will merge all them appropriately.
If you use a counts tab-delimited text file from Seurat as input, include the option:
--dataFormat counts
If more than 20k cells I suggest:
--downsample 5000
If the data is already scaled and log2, you can exclude the counts command. When the data is in sparse matrix format it knows to consider it counts.
Best,
Nathan

On Dec 14, 2020, at 6:14 PM, therealgenna<EMAIL_ADDRESS>wrote:

Great, thank you so much! I guess I'll have to extract count matrices and make text files from Seurat objects which were made/merged from multiple 10X run outputs.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

I would still recommend using the GitHub version which addresses a few bugs present in the last pypi release. Much more stable.
The zip files you mentioned are correct. The site-packages path can’t be modified so those folders are extracted to your root user directory and should be looked at by the program first when running which is the expected behavior for the PyPi. Not sure what the issue is yet.
Best,
Nathan

On Dec 14, 2020, at 9:04 PM, therealgenna<EMAIL_ADDRESS>wrote:

FYI, I noticed that with pip install I get Config.zip and AltDatabase.zip in the /path/topython2.7/site-packages/altanalyze but there are no unzipped versions. When I try to install the database as above, the ~/altanalyze is created in my home directory, under which I also get ~/altanalyze/Config and ~/altanalyze/AltDatabase, but then I get the reported error.
Moving Config and AltDatabase from ~/altanalyze to /path/topython2.7/site-packages/altanalyze works (and I guess unzipping the zip files would work as well).
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.

Awesome, I will follow your recommendations. In the end, after testing it on smaller sets, I might run it on ~400k cells. Will it work? What number should I use for downsampling?
Another question - I've emailed you a day or two ago asking about Louvain resolution in your ICGS2 paper. I wonder if you had a chance to see my email.
Thanks again!

Sorry for missing this before. Does the below help (ICGS_NMF.py script):
from annoy import AnnoyIndex
t=AnnoyIndex(nm,metric="euclidean")
for i in range(nn):
    try:  t.add_item(i,X[i])
    except Exception: print i
t.build(100)
diclst={}
for i in range(nn):
    ind=t.get_nns_by_item(i,10)
    diclst[i]=ind
G=nx.from_dict_of_lists(diclst)
dendrogram= community.generate_dendrogram(G)
level=0
pr= community.partition_at_level(dendrogram,level)


Multiple configuration files created

Describe the bug
The role this is dependent on (robertdebock.hashicorp) seems to create a config file under /etc/vault.d/vault.hcl where as this role creates and uses the config file /etc/vault/config.hcl. It might be cleaner to have one config file or at least have them under the same directory.
I would suggest taking a backup of /etc/vault.d/vault.hcl if it exists and overwriting it. I can send a pull request if you'd like to do it that way.
Playbook
My playbook consists of:
---
- hosts: all

  vars_files:
    - vars/main.yml

  roles:
    - robertdebock.bootstrap
    - robertdebock.hashicorp
    - robertdebock.vault

My vars/main.yml consists of:
---
bootstrap_user: vagrant

# robertdebock.hashicorp variables
hashicorp_products:
  - vault

# robertdebock.vault variables
vault_storages:
  - name: raft
    path: /opt/vault/data
    node_id: node1

Output
Show at least the error, possible related output, maybe just all the output.
Environment

Control node OS: MacOS Catalina 10.15.7
Control node Ansible version: ansible 2.9.11
Managed node OS: Ubuntu 20.04.1 LTS

Please consider sponsoring me.

Fully agreed. I'll work on getting the configuration in one directory.
Thanks again @rohankrishnadev!

I'll let CI test the changes and release a new version later, likely tomorrow.
Thanks for the heads up, certainly an improvement!

Just released version 3.0.0


Add appropriate favicon

We need to have a favicon!

[x] I ensure that this issue is not a duplicate of any current open issues(s).
[ ] Assign this issue to me (I want to work on this).
[ ] I will send my PR only after I'm assigned to this issue by any maintainer.
[ ] I understand that I will be unassigned from this issue if I fail to send a PR for this within a week.

Maybe a pokeball? @anshumanv

Of course!
As present in the body.

assign it please 😄

Assigned to Monal5031

@anshumanv its not a gh bug but only contributors can be assigned AFAIK, also I think the is not effective on the webapp can you check it once?

Yes, got it, correct

It will be functional by tonight.

@Monal5031 awesome favicon! Please check. :smile:


Docs CSC or CSC Docs?

Is it "Docs CSC" or "CSC Docs"? My vote for "Docs CSC" since it is docs.csc.fi :)

You mean like on the front page?
I would say we use either Docs CSC or then some complete form, e.g.
CSC Documentation.

CSCdocs

Poll situation (see here):
+2 CSC Docs
+2 Docs CSC

Stagnant


Content-Length Default Value Problem

In my program I have a case where th Content-Length received from a server can be -1 which means corrupted file or some problem. But when I am testing with MockServer, it does not allow me to change or remove the content-length header. It adds to the list as a second value even though I do not send  any body. Is there a way to change this value ? or create a response where the Content-Length header is removed ?

The problem is Content-Length is part of the HTTP specification so if I remove it from responses then the MockServer will return in valid HTTP responses.  I could potentially add a feature where if the Content-Length is already specified in the mock response I don't add a Content-Length header or update the existing one that way you could mock any Content-Length header.  I guess you only want this for the MockServer and not for the proxy?

I've committed a fix so I'm closing this ticket, once I've fixed a couple more bugs I'll do a full release.


Slow operations and cannot remove volume

Plugin version (or commit ref) : rclone:latest (ID from docker inspect 4ed13fa184fb0b79afa2a5d2f301764a1705ee7e8bb108c473726abfe4185085 PluginReference docker.io/rclone/docker-volume-rclone:amd64)
Docker version : 20.10.7, build f0df350
Plugin type : legacy/managed
Operating system: ubuntu 20.04

Description
I create a service which spawns some volumes managed via rclone. I use CEPH as an S3 backend.
When creating the volume the options look something like this:
"VolumeOptions": {
    "DriverConfig": {
        "Name": "rclone",
        "Options": {
            "allow-other": "true",
            "dir-cache-time": "10s",
            "path": "master-simcore/e5751e46-8f09-11ec-a814-02420a041bec/50b5e822-3c89-4809-aebe-03302ed656a6/home_jovyan_work_workspace",
            "poll-interval": "9s",
            "s3-access_key_id": "****************",
            "s3-endpoint": "https://ceph_endpoint_address",
            "s3-location_constraint": "",
            "s3-provider": "Minio",
            "s3-region": "us-east-1",
            "s3-secret_access_key": "****************",
            "s3-server_side_encryption": "",
            "type": "s3",
            "vfs-cache-mode": "minimal"
        }
    }

I've just noticed that the s3-provider is set to Minio and not Ceph. Maybe this is what is causing all the issues.
Below operation take a very long time each
$ docker volume ls
DRIVER          VOLUME NAME
rclone:latest   dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace
....

$ docker volume inspect dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace
[]
Error response from daemon: get dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace: error while checking if volume "dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace" exists in driver "rclone:latest": Post "http://%2Frun%2Fdocker%2Fplugins%2F4ed13fa184fb0b79afa2a5d2f301764a1705ee7e8bb108c473726abfe4185085%2Frclone.sock/VolumeDriver.Get": context deadline exceeded

$ docker volume rm -f  dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace
Error response from daemon: get dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace: error while checking if volume "dy-sidecar_1cd462bc-b958-467f-ac61-09d2faad6b07_home_jovyan_work_workspace" exists in driver "rclone:latest": Post "http://%2Frun%2Fdocker%2Fplugins%2F4ed13fa184fb0b79afa2a5d2f301764a1705ee7e8bb108c473726abfe4185085%2Frclone.sock/VolumeDriver.Get": context deadline exceeded

How would I remove a volume in such a situation?
Logs

Tests

I can provide more context now. Since I can reproduce this reliably. I have 10 volumes attached to 10 different services. When I remove the 10 docker swarm services and also delete the networks I get into the above situation. I try to remove the volume as soon as the service is removed.


Support Host Initiator Group(cluster) volume mapping - server part

Currently, the volume mapping in MIQ only supports mapping a volume to a single host. It is also important to provide option to allow mapping the volume to a cluster itself.
This PR is about server part.
UI part should provide a radio button to allow users to "map volume to host" or "map volume to cluster"
Depending on the option user selects display the list of hosts or cluster to be associated with a volume
part of https://github.com/ManageIQ/manageiq-providers-autosde/issues/93

@agrare done


Row selection - scope set to currentPage

Docs mention that we can set scope to currentPage,
When trying out row selection example and passing handlers selectAll method object { selectBy: "id", scope: "currentPage" } and testing it out, it seems like it is not working as expected.
Try clicking on Th checkbox and switching pages, or deselecting checkbox on single row it seems like it doesn't update as expected.

Thx for reporting it.
It is indeed broken, I didn't cover all use-cases for row selection.
I can fix it quick but i wonder what should happen if the user change the current page?

Deselect rows on previous page ?
Keep selection on previous page ?

Any idea welcomed

Depending on what is better for UX, not sure what would be better but:

if selection is kept on previous page - it might be confusing for users when sorting columns to keep track on selection

Works in version 1.12.2 as you have mentioned :100:

Thanks, you are right.
I leave as is for now 👍


Feature/270 expanding dftool ve functionality

Adding functionality to more easily interact w/ ve functions
Fixes https://github.com/oceanprotocol/df-py/issues/270

Will add tests and make sure they are passing as part of improving dftool functionality to simplify various interactions.
Changing this to draft until this is in place.

This PR needs a github issue that states the problem.
(Every PR should have a related github issue. And where possible it should be 1:1. Simpler.)

It does, it's issue https://github.com/oceanprotocol/df-py/issues/270

Resolved all issues and moved everything to other PR's.
I'd like to close this PR and delete any branches associated with it.

Can you pull from main and resolve the conflicts?

Can you pull from main and resolve the conflicts?
This PR is not supposed to be merged. Closing/deleting.


fix: rp tokens breaking other rules

When using rp option, other inline rules used around the ruby text, including builtin ones, are breaking. This PR fixes the problem. I've also included a test that fails with version 1.1.1 and is fixed by this PR.
Thank you for the plugin and for reviewing the changes!

Thank you for this valuable contribution! The fix for the nesting level issue and the additional test case make the plugin more robust. I've merged the changes and will release a new version (v1.1.2) shortly.


fix(@tsed/socketio): avoid rollup bundling errors (#2165)

By default, webpack and babel use the cjs export of socket.io, while rollup uses the esm export; Use aggregate import instead of default import to fix bundling errors caused by this difference.
Information

Type
Breaking change

Fix
No

Todos

[ ] Tests
[ ] Coverage
[ ] Example
[ ] Documentation

All tests are valid. Let’s go to merge ;)


Try to apply the example of Threejs's ocean, but the texture of the mouse scrolling flow will move with it.

Try to apply the example of Threejs's ocean, but the texture of the mouse scrolling flow will move with it.

There is no problem with a separate threejs example.Can u help me?
` vertexShader: [
"uniform mat4 textureMatrix;",
"uniform float time;",
  "varying vec4 mirrorCoord;",
  "varying vec4 worldPosition;",

  "#include <common>",
  "#include <fog_pars_vertex>",
  "#include <shadowmap_pars_vertex>",
  "#include <logdepthbuf_pars_vertex>",

  "void main() {",
  "	mirrorCoord = modelMatrix * vec4( position, 1.0 );",
  "	worldPosition = mirrorCoord.xyzw;",
  "	mirrorCoord = textureMatrix * mirrorCoord;",
  "	vec4 mvPosition =  modelViewMatrix * vec4( position, 1.0 );",
  "	gl_Position = projectionMatrix * mvPosition;",

  "#include <beginnormal_vertex>",
  "#include <defaultnormal_vertex>",
  "#include <logdepthbuf_vertex>",
  "#include <fog_vertex>",
  "#include <shadowmap_vertex>",
  "}"
].join("\n"),`

`  fragmentShader: [
"uniform sampler2D mirrorSampler;",
"uniform float alpha;",
"uniform float time;",
"uniform float size;",
"uniform float distortionScale;",
"uniform sampler2D normalSampler;",
"uniform vec3 sunColor;",
"uniform vec3 sunDirection;",
"uniform vec3 eye;",
"uniform vec3 waterColor;",
  "varying vec4 mirrorCoord;",
  "varying vec4 worldPosition;",

  "vec4 getNoise( vec2 uv ) {",
  "	vec2 uv0 = ( uv / 103.0 ) + vec2(time / 17.0, time / 29.0);",
  "	vec2 uv1 = uv / 107.0-vec2( time / -19.0, time / 31.0 );",
  "	vec2 uv2 = uv / vec2( 8907.0, 9803.0 ) + vec2( time / 101.0, time / 97.0 );",
  "	vec2 uv3 = uv / vec2( 1091.0, 1027.0 ) - vec2( time / 109.0, time / -113.0 );",
  "	vec4 noise = texture2D( normalSampler, uv0 ) +",
  "		texture2D( normalSampler, uv1 ) +",
  "		texture2D( normalSampler, uv2 ) +",
  "		texture2D( normalSampler, uv3 );",
  "	return noise * 0.5 - 1.0;",
  "}",

  "void sunLight( const vec3 surfaceNormal, const vec3 eyeDirection, float shiny, float spec, float diffuse, inout vec3 diffuseColor, inout vec3 specularColor ) {",
  "	vec3 reflection = normalize( reflect( -sunDirection, surfaceNormal ) );",
  "	float direction = max( 0.0, dot( eyeDirection, reflection ) );",
  "	specularColor += pow( direction, shiny ) * sunColor * spec;",
  "	diffuseColor += max( dot( sunDirection, surfaceNormal ), 0.0 ) * sunColor * diffuse;",
  "}",

  "#include <common>",
  "#include <packing>",
  "#include <bsdfs>",
  "#include <fog_pars_fragment>",
  "#include <logdepthbuf_pars_fragment>",
  "#include <lights_pars_begin>",
  "#include <shadowmap_pars_fragment>",
  "#include <shadowmask_pars_fragment>",

  "void main() {",

  "#include <logdepthbuf_fragment>",
  "	vec4 noise = getNoise( worldPosition.xy * size );",
  "	vec3 surfaceNormal = normalize( noise.xyz * vec3( 1.5, 1.0, 1.5 ) );",

  "	vec3 diffuseLight = vec3(0.0);",
  "	vec3 specularLight = vec3(0.0);",

  "	vec3 worldToEye = eye-worldPosition.xyz;",
  "	vec3 eyeDirection = normalize( worldToEye );",
  "	sunLight( surfaceNormal, eyeDirection, 100.0, 2.0, 0.5, diffuseLight, specularLight );",

  "	float distance = length(worldToEye);",

  "	vec2 distortion = surfaceNormal.xy * ( 0.001 + 1.0 / distance ) * distortionScale;",
  "	vec3 reflectionSample = vec3( texture2D( mirrorSampler, mirrorCoord.xy / mirrorCoord.w + distortion ) );",

  "	float theta = max( dot( eyeDirection, surfaceNormal ), 0.0 );",
  "	float rf0 = 0.3;",
  "	float reflectance = rf0 + ( 1.0 - rf0 ) * pow( ( 1.0 - theta ), 5.0 );",
  "	vec3 scatter = max( 0.0, dot( surfaceNormal, eyeDirection ) ) * waterColor;",
  "	vec3 albedo = mix( ( sunColor * diffuseLight * 0.3 + scatter ) * getShadowMask(), ( vec3( 0.1 ) + reflectionSample * 0.9 + reflectionSample * specularLight ), reflectance);",
  "	vec3 outgoingLight = albedo;",
  "	gl_FragColor = vec4( outgoingLight, alpha );",

  "#include <tonemapping_fragment>",
  "#include <fog_fragment>",
  "}"
].join("\n")`

@datouzhan , thanks for the issue.
What do you mean by:

Try to apply the example of Threejs's ocean, but the texture of the mouse scrolling flow will move with it.

Do you want the water texture to somehow follow the mouse?

i dont want the water texture to somehow follow the mouse ,but it follow.It confuse me

At present, the water texure will follow the mouse to zoom in and out.I dont want it follow.

@datouzhan , I can't see the issue from the gif, can you please upload a video somewhere and make it a bit longer, the gif is too stuttering

https://user-images.githubusercontent.com/20176393/117934234-0a288b80-b335-11eb-9493-f01d8da4b34c.mp4

Can you see the issue?

@datouzhan , yes, thank you, now it is much clearer, and sorry for the delay.
What value are you passing in as the "eye" value? I mean uniform vec3 eye;.

 var eye =     options.eye !== undefined ? options.eye : new THREE.Vector3(0, 0, 0); I
set the eye value to the new vector3,and then In onbeforeRender method, I set eye.setFromMatrixPosition(camera.matrixWorld);

Ok, thanks @datouzhan. Which camera do you use? I guess the .camara from the MapView ?

yes, the camera from thre mapview

I have an idea.
Can you try just removing that line. i.e. set the eye to 0,0,0 every frame.
The position of all objects is positioned relative to the camera, i.e. the camera is positioned at 0,0,0 and everything is rendered relative to it, to ensure the precision of the depth buffer.

Do you mean to delete the eye.setFromMatrixPosition(camera.matrixWorld); from beforeerender?

Yes

i try to delete the eye.setFromMatrixPosition(camera.matrixWorld); But the problem is still not solved

What does it look like? Can you please create another video?

I noticed also you have the following
worldPosition = mirrorCoord.xyzw;
mirrorCoord = textureMatrix * mirrorCoord;
vec4 mvPosition =  modelViewMatrix * vec4( position, 1.0 )

Can you instead set worldPosition to mvPosition?
Otherwise I don't know what the issue is. You will have to debug it step by step.

 "	mirrorCoord = modelMatrix * vec4( position, 1.0 );",
      // "	worldPosition = mirrorCoord.xyzw;",
      "	mirrorCoord = textureMatrix *  vec4( position, 1.0 );",
      "	vec4 mvPosition =  modelViewMatrix * vec4( position, 1.0 );",
      "	worldPosition = mvPosition;",
      "	gl_Position = projectionMatrix * mvPosition;",

i instead set the worldPosition to mvPosition,
now it look like
https://user-images.githubusercontent.com/20176393/118776635-c72d6180-b8ba-11eb-8f72-17d56b0ee363.mp4
Do you have an email? I can send you a simple example

Sure, please send to<EMAIL_ADDRESS>

hey @nzjony ,i am in the slack.My username is jane, can you find me？

@datouzhan , did you solve the issue in the end with the water?


The flutter inspector shows nothing

Steps to reproduce

Run Applicatioin in Android Studio/Intellij idea/VS Code
Open Flutter Inspector(Widget Inspector in VS Code)

The inspector's window is empty and shows nothing
Android Studio/Intellij idea:

VS Code:

The problem is observed not only on the Linux build, but also on android, web
Actual results
The flutter inspector shows nothing in Android Studio, Intellij idea, VS Code
Logs

Logs

<!-- Paste your logs here -->

Flutter Doctor output

Doctor output

Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel stable, 3.10.4, on Linux Mint 21 5.15.0-71-generic, locale
    ru_RU.UTF-8)
[!] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
    ! Some Android licenses not accepted. To resolve this, run: flutter doctor
      --android-licenses
[✓] Chrome - develop for the web
[✓] Linux toolchain - develop for Linux desktop
[✓] Android Studio (version 2022.3)
[!] Android Studio (version 2023.1)
    ✗ Unable to find bundled Java version.
[✓] IntelliJ IDEA Ultimate Edition (version 2022.3)
[✓] VS Code (version 1.78.0)
[✓] Connected device (2 available)
[✓] Network resources

@Lonti6
I verified on latest stable with Android Studio which shows inspector widget / details properly:

You are on older version of stable. Please upgrade and try again. Most probably you are hitting https://github.com/flutter/devtools/issues/4388.
Closing from here.


no action sequences included in the input

Thank you for the rtx1 implementation!
If I understand it correctly, the input does not include the corresponding action information for each frame. Therefore, when predicting future actions, one cannot refer to the historical action sequence.
My question would be how much does this affect the results?

Upvote & Fund

We're using Polar.sh so you can upvote and help fund this issue.
We receive the funding once the issue is completed & confirmed by you.
Thank you in advance for helping prioritize & fund our backlog.

@schwgHao well, the output now is logits of the interleaved images with text. It outputs text logits to be tokenized like llama.
Basically, to add the multi dimensional output as in the paper you would need to integrate token learner. There is an implementation in my zeta library if you would like to work make the model output 7 dimensional tensors.
https://zeta.apac.ai/en/latest/zeta/nn/modules/token_learner/


showCallout not working

Hi, I am trying to show a custom callout when the map is rendered.
<MapView.Marker 				ref={(MarkRef) => { 					if (MarkRef != null) { 						MarkRef.showCallout(); 					} 				}}
In Android, a callout is shown but it is empty (When I don't use showCallout() and press the marker it looks fine)
In iOS, no callout is opened

I had the same issue. What version of react-native-maps are you using?

Still having the issue on RN0.39.0 and RNM0.12.4

+1 also having this issue. I noticed that it doesn't work if you add "tooltip" to the callout like this:
<MapView.Callout tooltip>...</MapView.Callout>
But if you remove the tooltip part, showCallout() works just fine. Any solutions?

@arnsa Not if I use showCallout()

Does not work w/ custom callouts either. If you tooltip is removed only the caret shows and none of the custom callout.

I also having this issue

its a issue with onPress not firing for the marker on android.

merging to #1872


New Azkaban DSL

The current way of describing flows using Java Properties-based .job files is not ideal. As flows become more complex, the number of job files increases, and it becomes increasingly difficult to manage and debug issues, such as unintentional circular dependencies. The DSL should also be easily extensible.
A new DSL for describing Azkaban flows has been on the Azkaban wishlist for a long time and will make Azkaban even more easy and productive for users. Creating a new DSL will also allow us to address issues that have been brought up, such as arithmetic expressions (#224) and being able to set a custom name for a flow (#41).
The code for processing the new DSL should also be reusable enough so that it can be integrated into an Azkaban Gradle plugin to validate Azkaban projects locally without having to deploy them.

github.com/linkedin/linkedin-gradle-plugin-for-apache-hadoop

@pranayhasan Yes, I am aware that this plugin exists, but IMO workflow code should not be implemented by the build system.


Add .node-version file basic support

Noticed that the .node-version convention is not currently supported.

[!NOTE]
This is my first Rust PR, if you have suggestion, tips or remarks 😉

[x] Tests are added/updated if needed

Just a small question, when I run :
cargo run -- build examples/node-node-version --name node-node-version

I got this error... It's not breaking the tests but its weird, do you have any idea ?
Details

20.52 unpacking source archive /nix/store/6rn31a96dly28rhgli9nninq7dsaky9c-node-v14.21.3.tar.xz
22.29 source root is node-v14.21.3
22.39 setting SOURCE_DATE_EPOCH to timestamp<PHONE_NUMBER> of file node-v14.21.3/vcbuild.bat
22.40 patching sources
22.40 updateAutotoolsGnuConfigScriptsPhase
22.42 Updating Autotools / GNU config script to a newer upstream version: ./deps/cares/config.sub
22.44 Updating Autotools / GNU config script to a newer upstream version: ./deps/cares/config.guess
22.44 configuring
22.46 fixing libtool script ./deps/cares/ltmain.sh
22.47 ./deps/cares/configure
22.50 configure flags: --prefix=/nix/store/m65ddhqa3i1c4fvykqfk8vkwsk32crkf-nodejs-14.21.3 --shared-libuv --shared-libuv-libpath=/nix/store/kfdvif4lanw5zfdbhnp7hrjlrsxbzal9-libuv-1.46.0/lib --shared-openssl --shared-openssl-libpath=/nix/store/fyic70v28145nnr4s3ff03svr5w6gync-openssl-1.1.1w/lib --shared-zlib --shared-zlib-libpath=/nix/store/i6w8i3ps7x9cyyyvfif55rf1plvprp2f-zlib-1.3/lib --with-intl=system-icu --without-dtrace
22.52 Node.js configure: Found Python 3.11.5...
22.52 Please use python3.10 or python3.9 or python3.8 or python3.7 or python3.6 or python3.5 or python2.7.
23.05 error: builder for '/nix/store/fs2bjgbwk2ivs0h9cljih0j9n25nxhxf-nodejs-14.21.3.drv' failed with exit code 1
24.42 error: 1 dependencies of derivation '/nix/store/8b4ixp1kx3v239p0k1m6rh46p99qjf0l-bf744fe90419885eefced41b3e5ae442d732712d-env.drv' failed to build
------

 1 warning found (use docker --debug to expand):
 - UndefinedVar: Usage of undefined variable '$NIXPACKS_PATH' (line 18)
Dockerfile:8
--------------------
   6 |
   7 |     COPY .nixpacks/nixpkgs-bf744fe90419885eefced41b3e5ae442d732712d.nix .nixpacks/nixpkgs-bf744fe90419885eefced41b3e5ae442d732712d.nix
   8 | >>> RUN nix-env -if .nixpacks/nixpkgs-bf744fe90419885eefced41b3e5ae442d732712d.nix && nix-collect-garbage -d
   9 |
  10 |
--------------------
ERROR: failed to solve: process "/bin/bash -ol pipefail -c nix-env -if .nixpacks/nixpkgs-bf744fe90419885eefced41b3e5ae442d732712d.nix && nix-collect-garbage -d" did not complete successfully: exit code: 100
Error: Docker build failed

Check out some of the commands in the makefile to get better errors, the default errors from running rust tests are not great
https://github.com/railwayapp/nixpacks/blob/main/Makefile

The error is happening because there are no prebuilt binaries for Node 14 because it is such an old version and it is attempting to build it from source and there are missing dependencies needed. Using a newer version of Node should work.

Hey @coffee-cup, I've updated the PR

The version in the example is now v22 (LTS)
Tests updated
Example node-node-version is working well


IGAPP-445: Hide Push Notifications Setting if Disabled

This pull request belongs to an issue on our bugtracker.
You can find it there by looking for an issue with the key which is mentioned in the title of this pull request.
It starts with the keyword IGAPP.

When deactivating location in LocationPermissionManager we have
if (buildConfig().featureFlags.fixedCity) {
return RESULTS.UNAVAILABLE
}
in PushNotificationManager we don't.
Maybe it would be nice to have it consistent?


Styled component loses props in typescript after second wrap

styled-components v2.1.2
typescript v2.4.2
Double wrapping of custom react component in styled function has component without origin props in the result.
export default class Icon extends React.PureComponent<IconProps, {}> {
  public render() {
    const { className, icon, style = {}, ...others } = this.props;
    const { id, viewBox } = icon.default;
    const size = viewBox.length ? viewBox.split(' ') : [];
    const iconStyle = Object.assign({width: `${size[2]}px`, height: `${size[3]}px`}, {}, style);

    return (
      <styled.IconWrap className={className} {...others} style={iconStyle}>
        <svg viewBox={viewBox} className={styled.svg}>
          <use xlinkHref={`#${id}`} />
        </svg>
      </styled.IconWrap>
    );
  }
}
...
export const InputIcon = styled(Icon)``;
...
export const IconErase = styled(InputIcon)``;

and finally
<IconErase { ...{ icon: iconErase } />
Expected result
No errors, IconErase component has same props as origin Icon component
Actual Result
Type '{ icon: IconSVG }' has no properties in common with type 'IntrinsicAttributes & IntrinsicClassAttributes<Component<Theme
dOuterStyledProps<WithOptionalTheme...'.

Component has only default themed styled component props (i.e.  theme, innerRef etc.)

Probably the same as the previous bug report: https://github.com/styled-components/styled-components/issues/1068

There is custom theme in my case, that is set by code
const {
  default: styled,
  css,
  injectGlobal,
  keyframes,
  ThemeProvider,
} = styledComponents as ThemedStyledComponentsModule<any> as ThemedStyledComponentsModule<Theme>;

export { css, injectGlobal, keyframes, ThemeProvider, ThemeProps, styled };

This is definitely a duplicate of #1068. Although it proves that there's different ways to reproduce that bug :) If you'd like you can copy over the description of your Theme interface there :wink:

This was an old issue, but I'm getting a similar error with v4. Is this related somehow? I shared my problem on SO: https://stackoverflow.com/questions/53724583/why-this-wrapped-styled-component-errors-has-no-properties-in-common-with


Add nsswitch.conf to Docker image

Description
Created nsswitch.conf to use locally defined translations before DNS. Copied to /etc/nsswitch.conf in the image.
Motivation and Context
Fixes #387
This resolves issues where specifying custom DNS translations (in /etc/hosts for example) take no effect.
How Has This Been Tested?
My current production workaround to the issue this PR is fixing was to create this file on the host and mount it to the container.
Checklist:

[ ] My change requires a change to the documentation or CHANGELOG.
[ ] I have updated the documentation/CHANGELOG accordingly.
[x] I have created a feature (non-master) branch for my PR.

Please assist me with the checklist - I'm not sure what steps I may need to do :)

Hey @JoelSpeed!
I've added a new line and updated the Changelog. Anything else I need to do?


Added a useful reference link

I'm including a link that discusses SQL server and I hope it will benefit your repository and help your readers learn more about the subject. I hope you will find this worthy. Thank you.

You could add disk link to articles list here https://github.com/ktaranov/sqlserver-kit/blob/master/Articles/README.md


Cannot remove thumb location in edit (TV Episode for sure)

What TMM version are you using? SNAPSHOT-3,0
release, pre-release, nightly, or directly from GitHub/branch? nightly
What is the actual behaviour? Clearing thumb field does not clear the set thumb location set for the episode
What is the expected behaviour? Clearing thumb field clears the set thumb location set for the episode
Steps to reproduce:

open the TV Episode edit window

Go to 'Details 2' tab
Clear any set value for the field "Thumb"
Click OK on the edit window

Open the edit window again

Go to 'Details 2' tab
Note the Thumb field has still retained the previously set value

Additional
Just a note this is related to the problems I was having with 3.0-SNAPSHOT and my previous issue about image not changing... (yes on Linux with 2.xx.xx the problem was file acess, but when i ran SNAPSHOT-3.0 on Windows i found the problem was also persisted by the fact that I was recycling temp thumbnail location (just to get it to copy the image edited to the folder next to the file (I reopened that issue and illustrated how to reproduce that....) Now I find it's impossible to clear whatever location is already set, so me using a single file location where i edit and save my files is apparently exposing that issue, and not being able to clear the set location makes me have to use a different edit and save location every time.

No not the file just the link to the file.
Currently when you select a file it sets the Thumb field to "file:/path/to/file" and copies the image. I use the same file for editing images. So I want to clear that after the image is copies to location so if I ever edit and change the image it doesn't register as not being changed (since the link will be the same) and the edited file is copied next to video file

okay, that has been fixed by my changes. please review tomorrow (v3)


Notifies everyone on server

It seems to be notifying all players on the server regardless of their location.
Otherwise an amazing script mate! One suggestion would be to add the coin flicking sound.
Keep up the great work mate!
Also the latest version to download is not in english for notifications.
Ok, I just tried the latest downloadable version and used the old config for english notifications. Its all working sweet now mate. Cheers!

Hello DJJackal,
thank you, I will continue. I just have a lot to do at the moment, so I apologize for the delay in replying. I will look at the error and improve it.
I'm working on a V2 with many more options anyway. It will be so that there is a real coin on the head and no 3D text (but the option to turn it on and off is still there). This will take some time though, as I'm busy elsewhere at the moment.
Of course it will also be free to download.
Regards


Refactor page headings

Preview is available here:http://laboratory-pr1165.previews.kube001.services.stellar-ops.com/


Error in ORT_RESULTS_SCANNER_FILE configuration

It seems the specification of the ORT_RESULTS_SCANNER_FILE variable is incorrect, which leads to empty web reports in case the evaluater is disabled.
https://github.com/driveblocks/ort-gitlab-ci/blob/main/.gitlab-ci.yml#L166
In addition, we faced the challenge that the ORT_DISABLE_ADVISOR, ORT_DISABLE_EVALUATOR and ORT_DISABLE_SCANNER variables have to be set explicitly to false for the toolkit to work correctly. This is hard to grasp from the description of the variables as they do not have a default value.
I'll prepare a pull request to propose a fix for both topics.

I believe this has been fixed with the recent merges; in any case, we're going to deprecate / archive this project soon in favor of another GitLab integration approach. So I'm closing this.


[hotfix][typo]Fix typo in UnsignedTypeConversionITCase

What is the purpose of the change
(For example: This pull request makes task deployment go through the blob server, rather than through RPC. That way we avoid re-transferring them on each deployment (during recovery).)
Brief change log
Fix typo in UnsignedTypeConversionITCase.
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end deployment with large payloads (100MB)
Extended integration test for recovery after master (JobManager) failure
Added test that validates that TaskInfo is transferred only once across recoveries
Manually verified the change by running a 4 node cluser with 2 JobManagers and 4 TaskManagers, a stateful streaming program, and killing one JobManager and two TaskManagers during the execution, verifying that recovery happens correctly.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

Maybe the failed e2e test is not related to this PR. cc @wuchong

Yes. I think it just been fixed 2 days ago: FLINK-21277.

Merging...


fromPrimitiveCasting and toPrimitiveCasting aren't consistent across [ValueObject] and [VogenDefaults]

Describe the bug
This:
[ValueObject<Guid>(toPrimitiveCasting: CastOperator.Implicit)]

Generates this:
public static implicit operator System.Guid(GatewayId vo)

Whereas this:
[assembly: VogenDefaults(toPrimitiveCasting: CastOperator.Implicit)]

Generates this:
public static implicit operator GatewayId(System.Guid value)

It makes no sense. This is a bug.
public static implicit operator GatewayId(System.Guid value) represents a cast "to" the primitive, so what [ValueObject] does is the correct behavior here.
Steps to reproduce
Outlined above.
Expected behaviour
[assembly: VogenDefaults(toPrimitiveCasting: CastOperator.Implicit)]

should do what
[ValueObject<Guid>(toPrimitiveCasting: CastOperator.Implicit)]

does.

Thanks for the bug report @aradalvand - well spotted. This has been fixed and will be available in the next release in a few hours.

@SteveDunn Thanks a lot for the quick fix!
(Hopefully 1K stars soon!)


log4j config needs update

I just ran into a full file system due to rundeck/rundeck#1175, which is fixed upstream and included in the latest  RPM, but lost due to the template overwriting it. The easy solution is to just update the template with this and other changes from the upstream but this will likely be a problem again in the future and, in general, continues to limit users' flexibility.
Before I start coding anything, I would like to propose several options:

Just update the damn template and move on
Add a parameter to override the log4j template
Add a parameter to disable managing the log4j file altogether and let the user do it on his own if necessary.
Switch to a more granular way of managing the file, using ini_setting or augeas resources, and not disturb the stock config more than necessary.

Option 4 is my preference, encapsulating with a defined type. (I would probably use ini_setting, since that's already in use and people are generally more comfortable with that.)

@wcooley option 4 is also my preference. However, would a template override be the most straightforward way to do this?

@jyaworski The template override would be the easiest, although I generally prefer to edit package-installed files rather than replace -- that way, if improvements are made upstream, you can get them automatically (depending on how they're packaged).
Rather than re-inventing log4j management, however, it might be worth looking into danielgil/log4j to see if it fits.

@wcooley all right. I'm not entirely familiar with log4j, but I think I can refactor it to produce the same output at the very least.
I agree with the use of an upstream module, but we need to find out what the most common log4j solution is. I'm hesitant to introduce dependencies when I don't need to.
I may look at this.

Proposed solution: https://github.com/puppet-community/puppet-rundeck/pull/161

thank y'all! i'll close this issue for now


MapKit Subspec 4.0.0 isn't compatible with iOS?

PromiseKit version: 4.0.0
Xcode version: 8.0 GM (8A218a)
Installation method:

[x] CocoaPods
[ ] Carthage
[ ] Other (please specify)

I'm trying to upgrade my app to Swift 3. I'm running into what seems to be a blocking issue with the PromiseKit MapKit subspec.
Full app source code: https://github.com/aaronbrethorst/onebusaway-iphone/tree/swift3
Podfile: https://github.com/aaronbrethorst/onebusaway-iphone/blob/swift3/Podfile
$ pod install
Analyzing dependencies
Pre-downloading: `SwiftMessages` from `https://github.com/aaronbrethorst/SwiftMessages.git`, branch `xcode8-gm`
[!] The platform of the target `OBAKit` (iOS 9.0) is not compatible with `PromiseKit/MapKit (4.0.0)`, which does not support `ios`.

K, figured it out, seems to be a bug in CocoaPods where specifying any deployment target in the subspec causes the deployment targets for any other platforms to be erased. Work around is to re-specify.

I will push 4.0.1 to CocoaPods as soon as I have determined the issue with #538.

In the meantime you should be able to use what we have with:
pod "PromiseKit", git: "https://github.com/mxcl/PromiseKit", commit: "d8d9d95"

Awesome, thanks @mxcl


Remove warning from assembly component when feature/joining errors occured

@obucklin & @chenkasirer - it's important to know that something went wrong (even though this is not a fatal error). we should not tailor the plugin for complete beginners.


[BUG] Can't Find Local Icons

Environment
Self-Hosted (Docker)
System
Ubuntu Server 24.04 - CasaOS Instance
Version
3.1.1
Describe the problem
I'm using a docker compose file to deploy a stack in portioner for Dashy
The docker compose file I have set up is as follows:
---
version: "3.8"
services:
  dashy:
    # To build from source, replace 'image: lissy93/dashy' with 'build: .'
    # build: .
    image: lissy93/dashy
    container_name: Dashy
    # Pass in your config file below, by specifying the path on your host machine
    volumes:
      - /DATA/AppData/Dashy/User-Data/item-icons:/app/user-data/item-icons
      - /DATA/AppData/Dashy/User-Data/conf.yml:/app/user-data/conf.yml
    ports:
      - 4000:8080
    # Set any environmental variables
    environment:
      - NODE_ENV=production
    # Specify your user ID and group ID. You can find this by running `id -u` and `id -g`
      - UID=1000
      - GID=1000
    # Specify restart policy
    restart: unless-stopped
    # Configure healthchecks
    healthcheck:
      test: ['CMD', 'node', '/app/services/healthcheck']
      interval: 1m30s
      timeout: 10s
      retries: 3
      start_period: 40s

I created this following the documentation.  I'm having issues accessing the mounted icons folder within Dashy itself.  Whenever I try to use either the syntax:
icon: ytstudiored.svg
or use
/app/user-data/item-icons/ytstudiored.svg
when trying to add a local icon, it still shows a broken icon image every single time.
I'm unsure what I'm doing wrong here, and I would like some help on this if possible!
Please let me know if you need anything else from me.  I'm still fairly new to self-hosting and docker, so it could be something simple I overlooked.
Thanks in advance,
--wittywilla
Additional info

Please tick the boxes

[X] You have explained the issue clearly, and included all relevant info
[X] You are using a supported version of Dashy
[X] You've checked that this issue hasn't already been raised
[X] You've checked the docs and troubleshooting guide
[X] You agree to the code of conduct

figured out my mistake.  I was following along the documentation for the .yml file and not the UI. If someone else is stuck, all they have to do is type in the file name, no need for the "icon: "


Can't locate object method "onclick" via package "Data::HTML::Element::Input"

On some of my smokers the test suite fails:
Can't locate object method "onclick" via package "Data::HTML::Element::Input" at /usr/home/cpansand/.cpan/build/2024021211/Tags-HTML-Element-0.07-4/blib/lib/Tags/HTML/Element/Utils.pm line 79.
# Looks like your test exited with 255 just after 1.
t/Tags-HTML-Element-Input/05-process.t .........
Dubious, test returned 255 (wstat 65280, 0xff00)
Failed 4/5 subtests

This probably happens if some of the prereqs (Data::HTML::Element::Input?) is too old (0.10 vs 0.11?).

Yes, fixed in 0.08 version


Use of payload with HTTP GET

In many examples, a payload is used with GET requests, for example
curl -G -iX GET 'http://localhost:1026/ngsi-ld/v1/entities/urn:ngsi-ld:TemperatureSensor:001' \
-H 'Link: <http://context/ngsi-context.jsonld>; rel="http://www.w3.org/ns/json-ld#context"; type="application/ld+json"' \
-d 'attrs=temperature'

I think this should be avoided and all data should included as query parameters. Note that according to HTTP 1.1 RFC

A payload within a GET request message has no defined semantics;  sending a payload body on a GET request might cause some existing implementations to reject the request

There is no payload body in the GET requests.
There's an HTTP header (Link) and that I believe is defined by JSON-LD.
But, of course, HTTP headers are always present in any HTTP request.

Yeah, but the problem seems to be with the body:

sending a payload body on a GET request might cause some existing implementations to reject the request
And there is no body.

You aren't seriously proposing to not have any URL parameters in a query, are you?
If you don't like it, NGSI-LD defines what we call "POST Query", where the method is POST and all query parameters go in a payload body.  Afaik, all NGSI-LD brokers implements that API endpoint.

OK, now I saw that the -G option  puts everything specified in -d in URL. Sorry :(

ok, misunderstanding.
Kind of expected that :)
It was too weird :)))


DO NOT MERGE - prototype of new CallSettings shape

The tests don't even build yet - I wanted to get early feedback on the shape of the API.
With the gRPC-based retry config we always specify the overall deadline as the RPC deadline, so there's only one set of initial/max/multiplier to consider - so I've inlined that into RetrySettings.
The overall deadline is also somewhat separate from retry settings, so I've extracted that to the top level, and removed both BackoffSettings and CallTiming.
I believe this will now be significantly easier to use, due to the much-reduced nesting.

This all looks good to me. I like the simplification of retry.

This is now ready for actual review. We expect that server-side streaming retry may be affected by https://github.com/googleapis/gax-dotnet/issues/367

Thanks - will check all of these next week :)


Unneeded Native libraries in Android app

Hello maintainers. Thanks for your work.
Please, when analyzing an app I am writing, I discovered that some native libraries were contributing to 41% the size of the app!
And the native libraries were the libsecp256k1-kmp Windows DLL and MacOS dylib.
Please, is there a way for me to remove those?
Thanks.

Can you please share more details: how do you build your app and how do you include the secp256k1 dependency ? Is it a Kotlin KMM project or a standard Android project ? or even better, can you provide a link to your project ? Android apps built with gradle should just need an .aar file which only contains native libs for android platforms (arm64 and x86_64)

Also @sstone the issue occurred after I upgraded the library version to 0.7.0, which is quite strange.

Oh, nevermind, the issue still exists in version 0.6.4 as well. And for some reason, when removing the proguard rules I added, the app works in debug mode but doesn't in release mode.

Please, any updates on how to solve this? Thanks.
cc @sstone

What does "it does not work" mean here ? Did your app ever work before ? I suspect that this is a build issue (resources included in a standard JAR should never end up in the actual app, this is what AAR are for) but don't really know how to help you. We do have an android version of phoenix that uses secp256k1-kmp (see https://github.com/ACINQ/phoenix) that you can have a look at.

@sstone in debug mode, it includes the code needed to load the library, but in release mode, it seems not to.

Also, as for removing the .DLL and .dylib files included in the app, is it due to KMP or Gradle? I can't really understand why it would include those files even though I specifically included the android JNI dependency.

Found the root cause! A library I was using, had a dependency on this library, and thus imported all these files. I will notify the maintainer. Thanks for your patience, @sstone. :pray:

Thanks, glad you solved the problem!

:pray:


map(buffer) bug

getting differing results in two seemingly identical patches, one cpu, the other noodles-based
could it be that some frames get skipped in the second version ?
walk on noise.zip

map(buffer) nodes in wrap mode happen to return values outside of the wrapped range
map(buffer) bug.zip

bjoren from vvvvorums has come up with a change to map.fxh
https://discourse.vvvv.org/t/map-3d-buffer-bug/19379/4
didnt occur to me to post in happy.fxh


slab auto-mover anti-favours slab 2

Apart from production loads I've not been able to reproduce this behaviour with a test, so this is just observed behaviour, for which I apologise because I dislike such bug reports too :(
We use memcached heavily for caching, where cache object sizes vary a lot, so most slabs get populated well, and typically we run memcached with full or near full cache, evicting continuously as a matter of course, which is fine.
But the sort of slab pattern shown below is one I've seen happen a lot.  This is from a long running 1.5.12 version, with pretty much default options so auto-move has seen a lot of action.
You'll see that poor old slab 2 is way out of line with the other slabs, evicting hard with very short retention time.
Interestingly I find that if I try to manually reassign pages with a "slabs reassign -1 2" it does not work well, slab 2 tends to get an extra page, and then often very shortly later gets it taken away again.  The best strategy I've found to work around this is to explicitly steal pages from the slab with the most pages.  But even then after a while slab 2 still tends to lose its pages again, only by switching off the auto-mover can I stop that, but that's not something I want to do, because the auto-mover is very useful as the object size distribution tends to move around over time.
Feels as if something is off with the auto-mover strategy here?
 #  Item_Size  Max_age   Pages   Count   Full?  Evicted Evict_Time OOM
  1      96B        65s       1      13      no        0        0    0
  2     120B       859s      12  101675      no<PHONE_NUMBER>      859    0
  3     152B   1659607s<PHONE_NUMBER>      no 85690307  1659505    0
  4     192B   1959105s    1488 8121066      no 121855414  1958008    0
  5     240B   1970157s    9306 40649963      no 532225918  1970139    0
  6     304B   1969892s   13207 45546584      no 772434412  1969788    0
  7     384B   1969879s   13592 37104025      no 262466561  1969847    0
  8     480B   1970931s   20927 45704566      no 255613273  1970931    0
  9     600B   1972349s   17570 30694788      no 190660089  1972349    0
 10     752B   1998325s   18878 26315931      no 170392332  1998325    0
 11     944B   1972610s   32811 36420208      no 201718623  1972610    0
 12     1.2K   1972439s   37971 33604334      no 173829153  1972439    0
 13     1.4K   2043535s   21788 15425901      no 82356826  2043534    0
 14     1.8K   2009197s    1495  843179      no 34658932  2009200    0
 15     2.3K    402141s      46   20746     yes 25126460   205116    0
 16     2.8K   2036542s    4102 1480821      no 35900304  2036541    0
 17     3.5K   1971126s   31257 9002015      no 69675613  1971126    0
 18     4.4K   1970435s   28465 6546949      no 81708667  1970435    0
 19     5.5K   1970291s   13225 2433399      no 51844206  1970292    0
 20     6.9K    759130s    1166  171401      no  6942599   759128    0
 21     8.7K   2448390s       2     235      no   214855  2455997    0
 22    10.8K  17642001s       2      75      no    90405   122249    0
 23    13.6K  17723915s       2     115      no 11653861   416221    0
 24    16.9K  17621517s       2      87      no 38943946   414306    0
 25    21.2K  17621958s       2      64      no 11276385   363682    0
 26    26.5K  17621939s       2      33      no   986909   337650    0
 27    33.1K  17627551s       1       2      no      429     5020    0

For completeness here's the stats output too:
localhost:11211   Field       Value
         accepting_conns           1
               auth_cmds           0
             auth_errors           0
                   bytes<PHONE_NUMBER>61
              bytes_read<PHONE_NUMBER>9024
           bytes_written<PHONE_NUMBER>81406
              cas_badval           0
                cas_hits           0
              cas_misses           0
               cmd_flush           0
                 cmd_get<PHONE_NUMBER>99
                 cmd_set<PHONE_NUMBER>9
               cmd_touch           0
             conn_yields    36403479
   connection_structures        1103
   crawler_items_checked<PHONE_NUMBER>566
       crawler_reclaimed<PHONE_NUMBER>
        curr_connections         649
              curr_items   343929103
               decr_hits           0
             decr_misses           0
             delete_hits           0
           delete_misses           0
         direct_reclaims<PHONE_NUMBER>
          evicted_active         233
       evicted_unfetched<PHONE_NUMBER>
               evictions<PHONE_NUMBER>
       expired_unfetched<PHONE_NUMBER>
             get_expired    55961700
             get_flushed           0
                get_hits<PHONE_NUMBER>19
              get_misses<PHONE_NUMBER>0
              hash_bytes<PHONE_NUMBER>
       hash_is_expanding           0
        hash_power_level          29
               incr_hits<PHONE_NUMBER>
             incr_misses        1199
                libevent 2.0.21-stable
          limit_maxbytes<PHONE_NUMBER>24
     listen_disabled_num           0
        log_watcher_sent           0
     log_watcher_skipped           0
      log_worker_dropped           0
      log_worker_written           0
       lru_bumps_dropped           0
     lru_crawler_running           0
      lru_crawler_starts     6696422
  lru_maintainer_juggles<PHONE_NUMBER>0
       lrutail_reflocked        3131
            malloc_fails           0
         max_connections        8192
           moves_to_cold<PHONE_NUMBER>8
           moves_to_warm<PHONE_NUMBER>
        moves_within_lru    37650113
                     pid       14923
            pointer_size          64
               reclaimed    61392198
    rejected_connections           0
            reserved_fds          20
           rusage_system 3122520.712456
             rusage_user 2968703.518491
   slab_global_page_pool           0
slab_reassign_busy_deletes           0
slab_reassign_busy_items<PHONE_NUMBER>
slab_reassign_chunk_rescues           0
slab_reassign_evictions_nomem<PHONE_NUMBER>
slab_reassign_inline_reclaim<PHONE_NUMBER>
   slab_reassign_rescues   969198347
   slab_reassign_running           0
             slabs_moved     1205988
                 threads           4
                    time<PHONE_NUMBER>
time_in_listen_disabled_us           0
       total_connections    41110118
             total_items<PHONE_NUMBER>8
              touch_hits           0
            touch_misses           0
                  uptime    24810089
                 version      1.5.12

Thanks! Interesting. I don't think much has changed in the mover algo since 1.5.12. That definitely looks odd.
The automover algorithm(s) have python implementations in the scripts/ directory of the release tarball. They're designed so you just run the script against an instance; it disables the internal algo and runs the python version, which should be identical. Do you have any interest in trying out this script to tune or look at verbosity prints to debug what's going on? Or at least see if it makes the same decisions as the C one.
Another thing is the new: https://github.com/memcached/mctester - which I'm extending this week. It's designed to help reproduce production workloads. Unless I get a repro workload or eyeball the bug there won't be too much I can do since I don't have access to your production :)
How often do you run into this?

I'd forgotten about the python version, that's a great idea, I'll have a play.
I'm seeing this quite a lot actually, got about 100 boxes with large memcached instances, of which about 20 are in this sort of state.

Should be easy to spot then, I hope. Lets get this figured out :) The verbose option should print a good amount of detail right off the bat. Only caveat is once you shut down the script I don't think it automatically re-enables the internal algo, so just remember to do that before walking off. The script does automatically disable it when starting up.
Also, running watch sysevents on the daemon has some but not many prints about when the internal algo is making decisions, for what it's worth.

So interestingly having added a little more verbose output to the python code, I'm coming to the conclusion that in part there is an artefact of the work load, but also the drawback of sliding windows.  What's happening is that load and evictions on slab2 for some reason are somewhat bursty, which means it can evict for 10 or 15 seconds, then stop evicting for next 30 seconds or so.
This means that the default 30 sliding window (on 1s intervals) gets fooled into thinking that even though slab2 is the youngest by far its not evicting enough to be a candidate for a move.  Then slab2 ends up being stuck in a somewhat vicious cycle that it can never break out of.  If I drop the window to 10 slots, still with a 1s interval, it actually looks much happier in our load case.
3 servers in the same load group where the 1st and 3rd are running python mover with 10 slot window are starting to look much happier compared to the 2nd (this is memcached display output for slab 2 only):
2     120B      5240s      22  181761      no<PHONE_NUMBER>     2885    0
2     120B          69s        4   22017       no<PHONE_NUMBER>       54    0
2     120B      5528s      18  149619      no 926377566     4007    0
I'll have a stab at reproducing this locally now and see if I can amend the the sliding window algorithm to be less easily fooled by bursty load.

Cool! Looking forward to your results. I'm pretty busy at the moment so can't investigate myself for a few days.
Surprisingly, I basically made up that page mover algo off the top of my head and we received almost no bug reports or feedback since then.
IIRC window size and ratio are starttime options for the daemon. Think there's a runtime cmd for it too.

Any luck?

The modifications I've made to the python version seem to be helping my use case a fair bit.  Essentially extends the rolling window logic to also capture the size of the evictions in each slot, and then at any point you can see over the window that if one slab has a significant proportion of the total evictions it becomes a candidate for a move.
Its probably a somewhat pathological case, where there is a huge burst of evictions on the slab followed by a fairly quiet period.  I guess that for lower slabs this is more of an issue because the ratio between entries and pages is higher.  A test for this case is to fill up all the memory using higher slabs, then send short bursts of writes for short periods at, say, slab2.  Previously it would never get more space than its initial page allocation because it never has sustained traffic to be flagged for moves, so would always be short time evicting.
The change to the python code is tiny, I'll have a look at a c version next.

I think an earlier version of the algo was weighting the evictions by chunks per page but that was hitting a different edge case. Struggling to remember but that was a long time ago.

Fwiw, feel free to just post up your python changes. I can deal with the C stuff if it's harder, or at least take a look at what's changed so far. :)
Thanks!

Hiya, got a bit buried in work, I did fork and start looking at the c version, but got sidetracked.  The attached patch on the python script is, however, now running on a few dozen of our instances and all are now in much better balanced state.  So for us this modification seems to be working well.
memcached-automove.patch.txt

@mjhags any chance you could quickly put a real PR up? It's hard to credit you properly otherwise.
finally giving this a stricter look. In the original code it would set the target based on there being evictions seen (at all) during half of the window. You preserved this by adding window_check_count to count total non-zero values. You then change the 'ev' stat in the window to be the ratio of evictions in that slab vs the total evictions seen in that window.
then, if more than 25% of the total evictions in that window are in that slab class, it gets targeted for a new page.
Did you also change the window size at all? I'm guessing this makes it respond more to bursts; since a lot of evictions within a single window would trigger a page move now.
I'll think about this a bit more, but I'll do the C bits if you just open a PR for the python one, for proper attribution.

https://github.com/memcached/memcached/pull/695 submitted. Yep that's the extra control, no change to the window required for me.  This basically sits alongside the existing logic as a secondary decision point.

@mjhags thanks! I've merged that into next and done the C bits, also pushed to next. Any chance you could test the branch before the next release?

Had a play and that works well for me.
I have a very basic test rig that runs a small memory, 64m, memcached, fairly slowly fills it with continuous writes on higher slabs, and a second writer pushes stuff in bursts to slab2.  Without the fix slab2 sits with 1-2 pages all the time and has a very short evict time, with the fix it pushes more pages into slab 2 and evict-time is fairly good for all slabs.  Bit belt-and-braces and crude, so not really ideal for a unit test framework.