Add ConditionPathExists=/etc/initrd-release to targets

I noticed that several of our units actually run twice, including
ignition-files.service (!) and ignition-ostree-mount-firstboot-sysroot.service.
Looking deeper into this, I discovered a bit of the initrd that
I didn't even know existed, which is initrd-parse-etc.service
that actually runs ExecStart=-/usr/bin/systemctl --no-block start initrd-fs.target
post switch root, after services that are part of that target
have already run (and then been shut down).
This adds a big wrinkle to my understanding of system bootup.
Then I noticed that initrd-fs.target has:
ConditionPathExists=/etc/initrd-release
which will be false post-switchroot.
The semantics here are...strange.  It seems like the goal is to ensure
that the rootfs is definitely still mounted, perhaps in
very complex rootfs scenarios?  In our case it will still be
mounted, we definitely don't want to rerun any of our Ignition
units.
Adding the same condition to our targets ensures that they
aren't re-run post switchroot.

xref https://lists.freedesktop.org/archives/systemd-devel/2019-December/043790.html

that actually runs ExecStart=-/usr/bin/systemctl --no-block start initrd-fs.target
post switch root, after services that are part of that target
have already run (and then been shut down).

Oh wow. That's... scary. It's weird though, that unit should only be running in the initrd itself (from bootup(7). CL actually used to have ordering wrt this in the past. E.g. ignition-files.service had:
Before=initrd-parse-etc.service
After=initrd-root-fs.target

Seems like a systemd regression put in in the real root by mistake?

I think there's some kind of racy thing going on. If I boot the FCOS I just built locally normally:
[root@coreos ~]# journalctl -b 0 | grep initrd-parse-etc.service
Dec 05 16:24:29 coreos systemd[1]: initrd-parse-etc.service: Succeeded.
Dec 05 16:24:31 coreos systemd[1]: initrd-parse-etc.service: Succeeded.
[root@coreos ~]# journalctl -b 0 | grep 'files passed'
Dec 05 16:24:29 coreos ignition[780]: INFO     : files: files passed
Dec 05 16:24:31 coreos ignition[780]: INFO     : files: files passed

If I stop the boot using rd.break and then resume:
[root@coreos ~]# journalctl -b 0 | grep initrd-parse-etc.service
Dec 05 16:25:17 coreos systemd[1]: initrd-parse-etc.service: Succeeded.
[root@coreos ~]# journalctl -b 0 | grep 'files passed'
Dec 05 16:25:17 coreos ignition[780]: INFO     : files: files passed

So I think this might be systemd getting confused somehow.

Ohhh, I think this might just be the journal getting confused. Notice the pid is the same here:
Dec 05 16:24:29 coreos ignition[780]: INFO     : files: files passed
Dec 05 16:24:31 coreos ignition[780]: INFO     : files: files passed

And it's the same in other tests that I've run. So I don't think ignition-files.service is actually running twice.

And -o json-pretty reveals why:
# journalctl -b 0 --grep="files passed" -o json-pretty
{
        "SYSLOG_FACILITY" : "3",
        "_EXE" : "/usr/bin/ignition",
        "_MACHINE_ID" : "f3a1c3b65c0e4e10bf5c511499720c51",
        "_PID" : "781",
        "_GID" : "0",
        "_CAP_EFFECTIVE" : "3fffffffff",
        "_SYSTEMD_SLICE" : "system.slice",
        "__REALTIME_TIMESTAMP" : "1575563615190378",
        "SYSLOG_IDENTIFIER" : "ignition",
        "__MONOTONIC_TIMESTAMP" : "6220168",
        "_COMM" : "ignition",
        "_STREAM_ID" : "fdf84b64a8f04861b4c996a44f047be8",
        "__CURSOR" : "s=e030078e2be04702951333e34e93cfa1;i=2f2;b=96c10fee3e024fe88621d9e9b77c2d61;m=5ee988;t=598f77d9abd6a;x=c30effcfe9d>
        "_BOOT_ID" : "96c10fee3e024fe88621d9e9b77c2d61",
        "MESSAGE" : "INFO     : files: files passed",
        "_SELINUX_CONTEXT" : "kernel",
        "_SYSTEMD_CGROUP" : "/system.slice/ignition-files.service",
        "PRIORITY" : "6",
        "_SYSTEMD_INVOCATION_ID" : "1951a787387740a5b7aa64cd67ff5559",
        "_TRANSPORT" : "stdout",
        "_CMDLINE" : "/usr/bin/ignition --root=/sysroot --platform=qemu --stage=files --log-to-stdout",
        "_HOSTNAME" : "coreos",
        "_SYSTEMD_UNIT" : "ignition-files.service",
        "_UID" : "0"
}
{
        "_TRANSPORT" : "kernel",
        "_MACHINE_ID" : "d8e40229bf814913af3886c2aaa33f95",
        "SYSLOG_IDENTIFIER" : "ignition",
        "_BOOT_ID" : "96c10fee3e024fe88621d9e9b77c2d61",
        "SYSLOG_PID" : "781",
        "PRIORITY" : "6",
        "MESSAGE" : "INFO     : files: files passed",
        "SYSLOG_FACILITY" : "3",
        "__REALTIME_TIMESTAMP" : "1575563617516057",
        "__CURSOR" : "s=e030078e2be04702951333e34e93cfa1;i=38e;b=96c10fee3e024fe88621d9e9b77c2d61;m=826636;t=598f77dbe3a19;x=a712cd15e29>
        "__MONOTONIC_TIMESTAMP" : "8545846",
        "_HOSTNAME" : "coreos",
        "_SOURCE_MONOTONIC_TIMESTAMP" : "6402014"
}

It's getting logged twice, once via journald, and once via the kernel buffer I think because of: https://github.com/coreos/ignition-dracut/blob/f6b3c1b90b9aa62942d8d17ec82d91500947b55e/dracut/99journald-conf/00-journal-log-forwarding.conf#L11

Hmm...this might explain why we're seeing "Missed [x] kernel messages" in the journal too - if we're in some cases logging recursively, maybe journald tries to suppress this type of cycle but can't always do it?

So I think what's happening there is that it depends when switch root happens vs the journal reads input from the kernel buffer. If we switch root fast enough, the new journald instance which doesn't have that knob will be reading the data that the pre-switch-root journald left while forwarding.

Wow, thanks for that investigation.  Trying this some more, it does seem like you're right, and I was just lucky with the race when I was testing out this patch.
But...hum.  Then I guess my question is why isn't that unit re-running our services?  [Going to add some sanity checking that it really isn't happening]
Is the reason that ConditionPathExists=/etc/initrd-release is used in a bunch of units to simply avoid breakage if they're somehow enabled by default in the real root?
I updated the commit message with the new findings, but basically I'd argue for merging anyways since this seems like a best practice.

I updated the commit message with the new findings, but basically I'd argue for merging anyways since this seems like a best practice.

I think I agree, though... let's just plaster it in all our units?

Is the reason that ConditionPathExists=/etc/initrd-release is used in a bunch of units to simply avoid breakage if they're somehow enabled by default in the real root?

This is a pure guess, but I think it's simply because the build system puts all the service files indiscriminately in the real root and the 01systemd-initrd module just fetches them from there.
One nice bonus of doing this is I'm pretty sure it's what allows e.g. systemctl status initd-parse-etc.service to work correctly post-switchroot. If we do this to our units too, we should be able to get that working as well.

(I can investigate this and do that switch as a follow-up if folks agree. I think it'd be a nice tiny QOL improvement.)

I think I agree, though... let's just plaster it in all our units?

Yeah.

https://github.com/coreos/ignition-dracut/pull/142


Issue installing it with pip

I tried running:
pip install fasttext
and got this error:
In file included from fasttext/fasttext.cpp:310:
fasttext/cpp/src/dictionary.h:16:10: fatal error: 'random' file not found
#include <random>
         ^
1 error generated.
error: command 'gcc' failed with exit status 1

----------------------------------------
Failed building wheel for fasttext

Any idea on how to fix this ?

hi @MatthiasRMS
Please take a look at #28 to solve this problem

I'm having the same issue,
CC=/usr/bin/gcc-5.4  pip install fasttext
is resulting in the same error.
Curiously building a standalone instance of fasttext outside of pip does work with the same setup.

@MatthiasRMS is the solution works for you?
@Peter-E-Lenz can you give us the output from stderr ?

`CC=/usr/bin/gcc-5.4
pip install fasttext 2>&1 >/dev/null | less`
results in

Failed building wheel for fasttext
Command "/Users/plenz/anaconda/bin/python -u -c "import setuptools, tokenize;file='/private/var/folders/qk/v9kxv7sj5ls3k89gwk6615zm0000gt/T/pip-build-71A_LC/fasttext/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/qk/v9kxv7sj5ls3k89gwk6615zm0000gt/T/pip-BemSc7-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/qk/v9kxv7sj5ls3k89gwk6615zm0000gt/T/pip-build-71A_LC/fasttext/

Failed building wheel for fasttext

@Peter-E-Lenz ahh, currently we don't upload the package in wheel distribution yet.
We will upload the wheel distribution ASAP

sudo port install gcc7
$ port select --list gcc
Available versions for gcc:
mp-gcc7
mp-llvm-gcc42
none (active)
$ sudo port select --set gcc mp-gcc7
Password:
Selecting 'mp-gcc7' for 'gcc' succeeded. 'mp-gcc7' is now active.
pip install fasttext

same error encountered, tried all the above comments, yet nothing changed

Hi @flybirp, can you give us the error output from your pip install fasttext?
Note that 📓  : currently we don't distribute the package in wheel format yet. So if you install the package in wheel format, it will error. Try to use the egg format
Related #47

@pyk  thanks for your answer, I'll follow the solution today and let you know if it works out

hi there:
Installing collected packages: fasttext
Running setup.py install for fasttext ... error
Complete output from command /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-f12cdwfh/fasttext/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-kzayzgpb-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.5
creating build/lib.macosx-10.6-intel-3.5/fasttext
copying fasttext/init.py -> build/lib.macosx-10.6-intel-3.5/fasttext
copying fasttext/model.py -> build/lib.macosx-10.6-intel-3.5/fasttext
running build_ext
building 'fasttext.fasttext' extension
creating build/temp.macosx-10.6-intel-3.5
creating build/temp.macosx-10.6-intel-3.5/fasttext
creating build/temp.macosx-10.6-intel-3.5/fasttext/cpp
creating build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src
/usr/bin/clang -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/fasttext.cpp -o build/temp.macosx-10.6-intel-3.5/fasttext/fasttext.o -pthread -funroll-loops -std=c++0x
In file included from fasttext/fasttext.cpp:310:
fasttext/cpp/src/dictionary.h:16:10: fatal error: 'random' file not found
#include
^
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-f12cdwfh/fasttext/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-kzayzgpb-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-f12cdwfh/fasttext/

by the way, I did install newer version of gcc with homebrew

@flybirp have you tried this?
CC=/path/to/your/latest-gcc-binary pip install fasttext

change the /path/to/your/latest-gcc-binary with the latest one

yes, by trying this, will gives me a different error
Installing collected packages: fasttext
Running setup.py install for fasttext ... error
Complete output from command /Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-dxzsbtml/fasttext/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-r3gt3sl3-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
creating build
creating build/lib.macosx-10.6-intel-3.5
creating build/lib.macosx-10.6-intel-3.5/fasttext
copying fasttext/init.py -> build/lib.macosx-10.6-intel-3.5/fasttext
copying fasttext/model.py -> build/lib.macosx-10.6-intel-3.5/fasttext
running build_ext
building 'fasttext.fasttext' extension
creating build/temp.macosx-10.6-intel-3.5
creating build/temp.macosx-10.6-intel-3.5/fasttext
creating build/temp.macosx-10.6-intel-3.5/fasttext/cpp
creating build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/fasttext.cpp -o build/temp.macosx-10.6-intel-3.5/fasttext/fasttext.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/interface.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/interface.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
In file included from fasttext/interface.cc:15:0:
fasttext/cpp/src/fasttext.cc: In function 'void cbow(Dictionary&, Model&, const std::vector&, double&, int32_t&)':
fasttext/cpp/src/fasttext.cc:134:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int32_t w = 0; w < line.size(); w++) {
^~~~~~~~~~~
fasttext/cpp/src/fasttext.cc:138:41: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (c != 0 && w + c >= 0 && w + c < line.size()) {
^~~~~~~
fasttext/cpp/src/fasttext.cc: In function 'void skipgram(Dictionary&, Model&, const std::vector&, double&, int32_t&)':
fasttext/cpp/src/fasttext.cc:152:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int32_t w = 0; w < line.size(); w++) {
^~~~~~~~~~~
fasttext/cpp/src/fasttext.cc:156:41: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (c != 0 && w + c >= 0 && w + c < line.size()) {
^~~~~~~
fasttext/interface.cc: In function 'void trainWrapper(int, char**, int)':
fasttext/interface.cc:192:39: warning: 'char* tmpnam(char*)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of tmpnam(3), it is highly recommended that you use mkstemp(3) instead. [-Wdeprecated-declarations]
std::string temp_file_name = std::tmpnam(nullptr);
^~~~~~
In file included from /usr/include/wchar.h:90:0,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/cwchar:44,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/iosfwd:40,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/ios:38,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/ostream:38,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/iostream:39,
from fasttext/interface.cc:2:
/usr/include/stdio.h:276:7: note: declared here
char tmpnam(char );
^~~~~~
fasttext/interface.cc:192:53: warning: 'char tmpnam(char)' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of tmpnam(3), it is highly recommended that you use mkstemp(3) instead. [-Wdeprecated-declarations]
std::string temp_file_name = std::tmpnam(nullptr);
^
In file included from /usr/include/wchar.h:90:0,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/cwchar:44,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/iosfwd:40,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/ios:38,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/ostream:38,
from /usr/local/Cellar/gcc6/6.1.0/include/c++/6.1.0/iostream:39,
from fasttext/interface.cc:2:
/usr/include/stdio.h:276:7: note: declared here
char *tmpnam(char *);
^~~~~~
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/args.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/args.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/dictionary.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/dictionary.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
fasttext/cpp/src/dictionary.cc: In member function 'void Dictionary::computeNgrams(const string&, std::vector&)':
fasttext/cpp/src/dictionary.cc:129:52: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (size_t j = i, n = 1; j < word.size() && n <= args.maxn; n++) {
^~~~~~~~~~
fasttext/cpp/src/dictionary.cc:134:13: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (n >= args.minn) {
^~~~~~~~~~
fasttext/cpp/src/dictionary.cc: In member function 'void Dictionary::initNgrams()':
fasttext/cpp/src/dictionary.cc:143:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (size_t i = 0; i < size_; i++) {
^~~~~
fasttext/cpp/src/dictionary.cc: In member function 'void Dictionary::initTableDiscard()':
fasttext/cpp/src/dictionary.cc:216:24: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (size_t i = 0; i < size_; i++) {
^~~~~
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/matrix.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/matrix.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/model.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/model.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
fasttext/cpp/src/model.cc: In member function 'real Model::hierarchicalSoftmax(int32_t)':
fasttext/cpp/src/model.cc:71:25: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
for (int32_t i = 0; i < pathToRoot.size(); i++) {
^~~~~~~~~~~~~~~~~
fasttext/cpp/src/model.cc: In member function 'void Model::findKBest(int32_t, std::vector<std::pair<float, int> >&)':
fasttext/cpp/src/model.cc:127:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (heap.size() == k && output_[i] < heap.front().first) {
~~~~~~~~~~^~
fasttext/cpp/src/model.cc:132:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (heap.size() > k) {
~~~~~~~~~~^
fasttext/cpp/src/model.cc: In member function 'void Model::dfs(int32_t, int32_t, real, std::vector<std::pair<float, int> >&)':
fasttext/cpp/src/model.cc:141:19: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (heap.size() == k && score < heap.front().first) {
~~~~~~~~~~^~
fasttext/cpp/src/model.cc:148:21: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if (heap.size() > k) {
~~~~~~~~~~^
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/utils.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/utils.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
fasttext/cpp/src/utils.cc: In function 'void utils::seek(std::ifstream&, int64_t)':
fasttext/cpp/src/utils.cc:72:10: warning: unused variable 'c' [-Wunused-variable]
char c;
^
/usr/bin/gcc -fno-strict-aliasing -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m -c fasttext/cpp/src/vector.cc -o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/vector.o -pthread -funroll-loops -std=c++0x
gcc: warning: x86_64 conflicts with i386 (arch flags ignored)
cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
g++-4.2 -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -g build/temp.macosx-10.6-intel-3.5/fasttext/fasttext.o build/temp.macosx-10.6-intel-3.5/fasttext/interface.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/args.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/dictionary.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/matrix.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/model.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/utils.o build/temp.macosx-10.6-intel-3.5/fasttext/cpp/src/vector.o -o build/lib.macosx-10.6-intel-3.5/fasttext/fasttext.cpython-35m-darwin.so
unable to execute 'g++-4.2': No such file or directory
error: command 'g++-4.2' failed with exit status 1
----------------------------------------

Command "/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5 -u -c "import setuptools, tokenize;file='/private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-dxzsbtml/fasttext/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-r3gt3sl3-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/72/mzywkdcd73l_9pd98pdgmw280000gn/T/pip-build-dxzsbtml/fasttext/

unable to execute 'g++-4.2': No such file or directory
error: command 'g++-4.2' failed with exit status 1

@flybirp now the problem is g++-4.2 is not available on your system.
Strange, it should work with: env CC=/path/to/your/latest-gcc-binary pip install fasttext

Also running into the "Failed building wheel issue"
C:\Users\hiddenuser\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python
\9.0\VC\Bin\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -Ic:\python27\include -I
c:\python27\PC /Tpfasttext/fasttext.cpp /Fobuild\temp.win32-2.7\Release\fasttext
/fasttext.obj -pthread -funroll-loops -std=c++0x
cl : Command line warning D9002 : ignoring unknown option '-pthread'
cl : Command line warning D9002 : ignoring unknown option '-funroll-loops'
cl : Command line warning D9002 : ignoring unknown option '-std=c++0x'
fasttext.cpp
C:\Users\hiddenuser\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python
\9.0\VC\Include\xlocale(342) : warning C4530: C++ exception handler used, but un
wind semantics are not enabled. Specify /EHsc
c:\users\hiddenuser\appdata\local\temp\17\pip-build-ucwunz\fasttext\fasttext\cpp/
src/utils.h(31) : error C2146: syntax error : missing ';' before identifier 'siz
e'
c:\users\hiddenuser\appdata\local\temp\17\pip-build-ucwunz\fasttext\fasttext\cpp/
src/utils.h(31) : error C4430: missing type specifier - int assumed. Note: C++ d
oes not support default-int
c:\users\hiddenuser\appdata\local\temp\17\pip-build-ucwunz\fasttext\fasttext\cpp/
src/utils.h(31) : error C4430: missing type specifier - int assumed. Note: C++ d
oes not support default-int
c:\users\hiddenuser\appdata\local\temp\17\pip-build-ucwunz\fasttext\fasttext\cpp/
src/utils.h(32) : error C2061: syntax error : identifier 'int64_t'
fasttext/fasttext.cpp(308) : fatal error C1083: Cannot open include file: 'std
int.h': No such file or directory
error: command 'C:\Users\hiddenuser\AppData\Local\Programs\Common\Microsof
t\Visual C++ for Python\9.0\VC\Bin\cl.exe' failed with exit status 2

Failed building wheel for fasttext
Running setup.py clean for fasttext
Failed to build fasttext

Hi @londonm, we don't distribute this package in wheel format yet

Closed, since the OP already install the fasttext successfully #54


Unexpected text node: SpecialButton. A text node cannot be a child of a .

Describe the bug
When using the Button-component in web together with the styled-function I get the following error:
Unexpected text node: SpecialButton. A text node cannot be a child of a <View>.
To Reproduce
import { Button, styled } from 'tamagui';

const SpecialButton = styled(Button, {
  color: 'red',
});

const Component = () => {
  <SpecialButton>SpecialButton</SpecialButton> /* <-- Gives error */
  /* <Button>Button</Button> */ /* <-- Works fine */
}

System Info (please complete the following information):
Output of npx envinfo --system --npmPackages '{vitest,vite,@vitejs/*}' --binaries --browsers
System:
  OS: macOS 12.4
  CPU: (10) x64 Apple M1 Max
  Memory: 418.04 MB / 64.00 GB
  Shell: 5.8.1 - /bin/zsh
Binaries:
  Node: 14.18.2 - /usr/local/bin/node
  Yarn: 4.0.0-rc.13 - /usr/local/bin/yarn
  npm: 8.5.3 - /usr/local/bin/npm
Browsers:
  Chrome: 104.0.5112.101
  Firefox Developer Edition: 104.0
  Safari: 15.5
  Safari Technology Preview: 15.4

In the latest versions it should warn that styled(Button) is not actually allowed. The way styled() works is it takes either a base React Native view, or another styled view, but not just any old component. Button is a functional component so it doesn't work, but I did add a warning recently that should say as much.
It's possible eventually to get this to work, I think, but it's more complicated than it may seem so I'm holding off for a while.


Use 4K cameras for better facial recognition?

Even though i use arcnet-cpu, and det-threshold of 98, i still get mislabeled faces outside. Partly this might be from needing more pictures (have about 80-100 per person, from different angles/light/etc) and partly because the resolution is to low (using 1080P) when viewed from far away.
Seriously thinking about upgrading to 4K cameras. Going from substream to main stream was such a big difference, and my inference speed event went down to 10ms on the coral TPU.

Hi. I have never received false recognition at the threshold of 0.98. I did this: 1 photo per person in the database. In good lighting, without shadows and glare, the shoulders should be visible. A look into the camera. Behind the person is a plain, light, simple background. Photos on a good iPhone camera without effects. I used Mobile Net. The recognition camera is set to good quality. There should be good lighting for better recognition. At a distance of several meters, I recognized without problems. There were a couple of false ones when there were photos of poor quality in the database... The higher the resolution of the camera, the more it recognizes from a greater distance. Can you write to me in Telegram

Interesting. Might have to try that, just one pic per person. Yes, i'm either going to point my camera downwards so it doesn't pick up everyone from the street anymore, and 'force' a good resolution, or get 4K cameras (albeit that would be an expensive upgrade, and no idea if i can run 6 of those at mainstream in frigate).
At 98% and with arcnet i do get good results, and unknowns, but every now and then it will still mislabel.
Maybe i will just get a 4 channel set and see how they perform, i have a feeling a lot of this mainly happen because of low resolution on recognition (because it's too far away, which 4K would help with. Same thing happened when i moved det_threshold to 98, it would not accept low res faces for recognition anymore, any blurry pics were discarded).

I am using double-take in front of compreface, I had to set  min_area: 1750 in doublet-take to stop feeding horrible low pixel blobs into detection. I can get away with a det_prob_threshold: 0.97 in compreface with this..
I think compreface needs a similar min area..  I found that sure I could recognize someone based on hare and body no problem but the faces are a mess past a certain threshold.
All training photos should be as high quality / pixel count as you can get.. I rarely train from anything other than photos from my iPhone etc, cropped a bit.. security footage is often blurred and the mort blur / soft your training photos the more broad the matches tend to be.
Going back to the original question.. You are going to need "more pixel density"  to capture faces at a distance.. 1080p will probably work at a doorbell cam for example because the person will be so close and filling the frame.. If you have a high mounted camera pointed down a hall way or outside.. You will need more resolution for the faces to even be distinguishable.


Enhancements to terra-application (loading overlay, error boundary, navigation prompt integration)

Summary
Adding a couple new APIs to terra-application to support our ongoing application framework development.
Documentation is bare-bones at the minute.  What's added here is the bare-minimum.

Can you also add the en-CA translation file?

We've decided to break terra-application out of the monorepo to head off concerns we have about maintainability long-term. Rather than merge this here, I'm going to lift this PR branch as the basis for the new package.


Update: Cilium version update

Description
Update of cilium version.
Issues:
Refs: #2015
How has this been tested:

[x] Covered by existing integration testing
[ ] Added integration testing to cover
[ ] Verified all A/C passes

[ ] develop
[ ] master
[ ] tag/other branch

[ ] Test environment

[ ] Shared Packet K8s cluster
[ ] New Packet K8s cluster
[ ] Kind cluster

[ ] Have not tested

Types of changes:

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation update

Checklist:
Documentation

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] No updates required.

Code Review

[ ] Does the test handle fatal exceptions, ie. rescue block

Issue

[x] Tasks in issue are checked off

Build running here - https://github.com/cnti-testcatalog/testsuite/actions/runs/9160350427
I'll post an update when the build is complete.


Use some kind of synchronization primitive when copying files for prebuilt detection

https://github.com/dotnet/installer/blob/4106448ffae0ac01c5cdec31608b7904aaf1ef05/src/SourceBuild/content/repo-projects/Directory.Build.targets#L495-L511 copies restored packages from the inner build to a global cache. Because many builds restore the same packages, it's very possible to race and cause locking issues.
Retries aren't a great solution here because they cause warnings, which can then get picked up as errors. Instead, if a global shared folder is desirable, a custom copy task should be used that handles synchronization between parallel build processes.

Rather than doing any fancy synchronization it might be easiest for each repo to output to it's own staging directory which are then consolidated after all repos are done building.

Rather than doing any fancy synchronization it might be easiest for each repo to output to it's own staging directory which are then consolidated after all repos are done building.

I guess this wouldn't work if using the clean-while-building option since the files would get deleted before it got to the end of the build.


enhance: generate CSVs for backported data

We would like to be able to use backported variables in explorers. Currently we do this in one of two ways. We either:

Create an ETL step for the explorer that generates CSVs
Link to a CSV from the auto-generated owid-datasets repo that Grapher publishes

This PR adds a third way, by publishing all backported data in CSV format.

On second thought this isn't a sensible approach, we'll stick with either using grapher charts or explicit explorer steps.


Broken Link

Hello, there are broken link in porting/examples/linux/README.md
https://mynewt.apache.org/network/ble/ble_intro/

Hi,
Thanks for reporting, fixed


Proper way to upgrade pip.

This seems to be recommended way to upgrade pip. (https://pip.pypa.io/en/stable/installing/)
The reason being, pip we use is being shipped with python 3.4 and we have to upgrade pip as soon as we encounter it first time.
The two ways to upgrade are

Download and install

curl -O https://bootstrap.pypa.io/get-pip.py && python get-pip.py

Or the current way.

Since this 2. way delegates the download to pip itself, this is better than explicit download and run.

Testing
Upgrade happens like this:
+ ./prep_local
Collecting pip
  Using cached pip-8.1.2-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.1
    Uninstalling pip-8.1.1:
      Successfully uninstalled pip-8.1.1
Successfully installed pip-8.1.2

:ship:


Work around runner SDK command-line tools problems

yuck
  ./gradlew connectedCheck
  Pre emulator launch script:
Install Android SDK
  /usr/bin/sh -c \sudo chown $USER:$USER /usr/local/lib/android/sdk -R
  /usr/bin/sh -c \yes | sdkmanager --licenses > /dev/null
  /usr/bin/sh: 1: sdkmanager: not found
Terminate Emulator
  /usr/local/lib/android/sdk/platform-tools/adb -s emulator-5554 emu kill
  error: could not connect to TCP port 5554: Connection refused
  The process '/usr/local/lib/android/sdk/platform-tools/adb' failed with exit code 1
Error: The process '/usr/bin/sh' failed with exit code 127


Unable to install from Git

Running go install github.com/jippi/dottie/cmd@main yielded this:
go: downloading github.com/jippi/dottie v0.11.3-0.20240507115018-9221955c336b
go: github.com/jippi/dottie/cmd@main (in github.com/jippi/dottie@v0.11.3-0.20240507115018-9221955c336b):
        The go.mod file for the module providing named packages contains one or
        more replace directives. It must not contain directives that would cause
        it to be interpreted differently than if it were the main module.

Ah; I guess that's broken currently due to some bug fixes in my forks that's pending upstream :)
For now you have to use one of the other installation options

Also; I need to update the docs go install github.com/jippi/dottie@main is the correct command after a refactor (maybe that works?)

Still got the same error with go install github.com/jippi/dottie@main


tstudio import fix

Added a function in the import flow to fix files imported from tStudio.  Fixes #206
Currently it has the following fixes.

\id lowercase to uppercase
Add new line after every verse
\toc3 lowercase to uppercase
\c remove text after chapter
add \p tag after every chapter
\ide and \ usfm correction

As per the team discussion this fix is not need to be incorporated with scribe.


Bump rustls to 0.17.0 and release next version

Released with 0.7.0.
https://crates.io/crates/async-tls


Organize and document

I would like to contribute @avincodes  :

[ ] Add Categories Based on Topic/ Langauge used
[ ] Add a Table of Contents with links to each topic
[ ] Add a node js code snippet (Show your Bussiness Card in the Terminal)

@BoshraJaber you can go for all 3!

i would like to contribute

go ahead @Shoyebaktar-shirol

Hello @avincodes  do i know more about the issue    Organize and document #81  what actually can i do , because i am a beginner in open source i hope you will help me .


ha-bestway is not a valid add-on repository

https://github.com/cdpuk/ha-bestway is not a valid add-on repository when trying to add to HACS

The HACS installation steps can be found here, and are still working for me.
https://hacs.xyz/docs/faq/custom_repositories
Can you describe the steps to reproduce the issue?


Create setup.py for different toil-scripts branches

Discussed today with hannes and for now separate branches are fine.


correct youtube video link

Thanks! I'm sure I tested that at the time...

Now sync'd to the website as well.


Here are the commands I am running in command line to run my scripts! Lets add them here for Readme

Rscript scripts/column_selection_script.R "data/Processed/selected_columns.csv"
Rscript scripts/clean_and_organize_script.R "data/Processed/selected_columns.csv" "data/Processed/clean_wrangled_titanic_data.csv"

Here is the command for my section
Rscript scripts/EDA_scripts.R --dataset=data/Processed/clean_wrangled_titanic_data.csv --out_dir=results/plots/


sus::error::Error

Using Result in any complex way suddenly requires good Error types.
How do we make errors that all share an API? Inheritance like Iterators probably.
We can use virtual more freely in error handling paths if needed.
We need error conversion as well, which is already possible through sus::ops::try_preserve_error
The Results returned in subdoc/lib/gen are a good example. The majority of functions return MarkdownToHtmlError but the root generate function needs to return either std::error_code wrapped in GenerateFileError or the MarkdownToHtmlError.
Secondly, there's a lot of boilerplate in returning results in this directory, which hopefully https://github.com/chromium/subspace/issues/299 can help with. But it will need to be able to convert the errors too. So all Errors should be convertible to some generic thing (Box<sus::error::DynError>).

Tasks

[x] Create the Error concept https://github.com/chromium/subspace/pull/328
[ ] Make our various errors into Error types (int conversion error, anything that we put in a Result error slot)
[ ] Make the top Subdoc generate functions return Box instead of a Choice type. The message generation gets to move into the ErrorImpl implementation. It could keep the underlying source Error too.

I think due to https://github.com/chromium/subspace/issues/333#issuecomment-1692369886 DynError has to go.
There is no way to downcast safely from a DynError to a specific error without RTTI, and we don't want to rely on RTTI. Large projects like Chromium and LLVM disable it.
Unlike Rust, C++ has its own way to express the type erasure and it's inheritance. So if an application wants to make its errors able to downcast it will need to provide its own type-erased base class.
Box<DynError> is special cased in Rust to construct from an Error but in C++ then what we need instead is that Box<SuperClass> can be constructed from Subclass, and there's no Error-ness involved at all there.
Then:

apps can opt into a generic error type and provide downcast traits for it as needed.
Error can still be applied to types you don't control like enums and std::error_code

Also the source() is problematic because it has to return a type-erased error to be part of the type-erased error API. So without DynError we could instead say that the source() just returns an Option<T: Error const&>, or we could just get rid of it and let the application domain put whatever api they want on their root error types as desired.
If we force the error type to return Option<T: Error const&> then the root generic type in an application needs to return itself which makes a cyclical dependency to resolve to determine if it satisfies Error.
As such, I think there's no room for DynError or for source() for the Error concept.

The thing to think about then is what if sus::foo() returned an Error and Subdoc wanted to return it from the source() of a GenerateError? What can it turn it into? Don't we need a DynError just for that scenario, passing errors across API boundaries? In that case you don't need to downcast anymore, it could be literally any error really. The downcast would be for debugging on the GenerateError, but not its source...?

Ok I think this is a matter of writing documentation explaining good practices. And explicitly not supporting Downcast for DynError
If you want to be able to get back your error type, you should make your own generic error base class and support Downcast from it. If you want to propagate a source error with more type information, then you should make your error types hold it in a well-typed way and give access to it. If you just want to pass along an error (and its sources) to read the messages from them, then DynError is fine.


Cannot load route from Red Hat Quarkus superHeroes example

Describe the Bug
Using route defined here https://github.com/pprosser-redhat/quarkus-super-heroes/blob/c530dd797f0ab35e50a0c07e6cf7cb4ab7c75a14/camelkfordebezium/crudlegacyheroes.camel.yaml#L3
- from:
      uri: kamelet:kafka-source
      id: from-4592
      parameters:
        bootstrapServers: my-cluster-kafka-bootstrap.superheroes.svc:9092
        topic: philsmac.phil.hero
        user: test
        password: test
        securityProtocol: PLAINTEXT
        consumerGroup: customers4
        autoCommitEnable: true
        autoOffsetReset: earliest
      steps:
        - log:
            message: '"Boom, a message ${body}"'
            id: log-9b74
            loggingLevel: INFO
        - choice:
            when:
              - id: when-6f94
                expression:
                  simple:
                    expression: ${body} != null
                    id: simple-6ea9
                steps:
                  - setHeader:
                      name: operation
                      id: setHeader-64d1
                      expression:
                        jsonpath:
                          expression: $.payload.op
                          id: jsonpath-dc29
                      description: get operation value
                  - setHeader:
                      name: snapshot
                      id: setHeader-1145
                      expression:
                        jsonpath:
                          expression: $.payload.source.snapshot
                          id: jsonpath-4564
                      description: get snapshot value
                  - to:
                      uri: direct:WorkOutWhatToDo
                      id: to-88d9
                      description: How do I talk to the DB
                  - log:
                      message: done
                      id: log-0068
                  - choice:
                      when:
                        - id: when-c579
                          expression:
                            simple:
                              expression: ${header.operation} == 'u'
                              id: simple-ae94
                          description: Update Hero
                          steps:
                            - to:
                                uri: kamelet:postgresql-sink
                                id: to-97da
                                parameters:
                                  query: >-
                                    UPDATE hero SET level = :#level, name =
                                    :#name, othername = :#othername, picture =
                                    :#picture, powers = :#powers  WHERE id =
                                    :#id
                                  databaseName: heroes_database
                                  serverName: heroes-db.default.svc.cluster.local
                                  serverPort: 5432
                                  username: superman
                                  password: superman
                                description: update DB
                        - id: when-6bc7
                          expression:
                            simple:
                              expression: ${header.operation} == 'c'
                              id: simple-8a4b
                          description: Create a new Hero
                          steps:
                            - to:
                                uri: kamelet:postgresql-sink
                                id: to-198b
                                parameters:
                                  query: >-
                                    INSERT INTO hero (id, level, name,
                                    othername, picture, powers) VALUES (:#id,
                                    :#level, :#name, :#othername, :#picture,
                                    :#powers)
                                  databaseName: heroes_database
                                  serverName: heroes-db.default.svc.cluster.local
                                  serverPort: 5432
                                  username: superman
                                  password: superman
                                  description: Insert DB
                                description: Insert DB
                        - id: when-24b0
                          expression:
                            simple:
                              expression: ${header.operation} == 'd'
                              id: simple-c4d6
                          description: Delete a Hero
                          steps:
                            - to:
                                uri: kamelet:postgresql-sink
                                id: to-522f
                                parameters:
                                  query: DELETE FROM hero WHERE id=:#id
                                  databaseName: heroes_database
                                  serverName: heroes-db.default.svc.cluster.local
                                  serverPort: '5432'
                                  username: superman
                                  password: superman
                                description: Delete DB
                      id: choice-40b3
            id: choice-f747
            otherwise:
              id: otherwise-41f6
              steps:
                - log:
                    message: no message to process
                    id: log-e37a
                    description: no message to process

it cannot be loaded
in console log:
VM475 react_devtools_backend.js:2655 TypeError: Cannot read properties of null (reading 'name')
    at stepsService.ts:466:27
    at Array.forEach (<anonymous>)
    at StepsService.regenerateUuids (stepsService.ts:465:14)
    at stepsService.ts:471:47
    at Array.forEach (<anonymous>)
    at stepsService.ts:469:24
    at Array.forEach (<anonymous>)
    at StepsService.regenerateUuids (stepsService.ts:465:14)
    at stepsService.ts:471:47
    at Array.forEach (<anonymous>)

Steps to Reproduce the Bug or Issue

Open source code editor
Paste route provided above
Click sync code
--> nothing is loaded

Screenshots or Videos
No response
Platform
OperateFirst

While trying to qualify and simplify the route to reproduce, I'm no more able to reproduce with the full route on OperateFirst (and can still reproduce on SV Code with 0.7.5) so i guess it was fixed


Freeze of GlobalSimilarity.py

Hi, when I run GlobalSimilarity.py it seems to run but it just freezes and nothing happens, this is the output:
root@0f7ecc515774:~/glosim2# python GlobalSimilarity.py ../100.xyz --nocenters 1
 WARNING: libAtoms::Hello World: environment variable OMP_STACKSIZE not set explicitly. The default value - system and compiler dependent - may be too small for some applications.
_____/\\\\\\\\\\\\__/\\\\\\_____________________________________________________________/\\\\\\\\\_____
 ___/\\\//////////__\////\\\___________________________________________________________/\\\///////\\\___
  __/\\\________________\/\\\________________________________/\\\______________________\///______\//\\\__
   _\/\\\____/\\\\\\\____\/\\\________/\\\\\_____/\\\\\\\\\\_\///_____/\\\\\__/\\\\\______________/\\\/___
    _\/\\\___\/////\\\____\/\\\______/\\\///\\\__\/\\\//////___/\\\__/\\\///\\\\\///\\\_________/\\\//_____
     _\/\\\_______\/\\\____\/\\\_____/\\\__\//\\\_\/\\\\\\\\\\_\/\\\_\/\\\_\//\\\__\/\\\______/\\\//________
      _\/\\\_______\/\\\____\/\\\____\//\\\__/\\\__\////////\\\_\/\\\_\/\\\__\/\\\__\/\\\____/\\\/___________
       _\//\\\\\\\\\\\\/___/\\\\\\\\\__\///\\\\\/____/\\\\\\\\\\_\/\\\_\/\\\__\/\\\__\/\\\___/\\\\\\\\\\\\\\\_
        __\////////////____\/////////_____\/////_____\//////////__\///__\///___\///___\///___\///////////////__
Sat Mar 17 08:33:35 2018
Start Computing the global average kernel of ../100.xyz
using output prefix = ../100-n8-l6-c3.5-g0.5-cw1.0-cotw0.5-average-zeta2-nonorm
Reading 100 input atomic structure from ../100.xyz: done 00:00:00 (h:m:s)
Compute soap and environmental kernels with a pool of 4 workers and 4 threads over 10 chunks: 00:00:00 (h:m:s)
listener  False
Env kernels:   0%|                                                                                | 0/5050 [00:00<?, ?it/s]

This is in the quippy docker and with 100.xyz contains 100 libatoms xyz crystal structures.
Do you maybe know what is going on?

Sometimes I get this error:
Exception AttributeError: "'tqdm' object has no attribute 'pos'" in <object repr() failed> ignored
Traceback (most recent call last):
  File "GlobalSimilarity.py", line 249, in <module>
    **soap_params)
  File "GlobalSimilarity.py", line 114, in get_environmentalKernels
    environmentalKernels = get_environmentalKernels_mt_mp_chunks(**kargs)
  File "/root/glosim2/libmatch/environmental_kernel.py", line 658, in get_environmentalKernels_mt_mp_chunks
    results = pool.run()
  File "/root/glosim2/libmatch/environmental_kernel.py", line 526, in run
    res = self.pool.map(self.func, self.chunks)
  File "/usr/local/lib/python2.7/multiprocessing/pool.py", line 253, in map
    return self.map_async(func, iterable, chunksize).get()
  File "/usr/local/lib/python2.7/multiprocessing/pool.py", line 572, in get
    raise self._value
AttributeError: 'tqdm' object has no attribute 'pos'

It might be related to https://github.com/tqdm/tqdm/issues/323 (which is still an open issue)

With 1 process/chunk/thread
get_environmentalKernels(islow_memory=False, nthreads=1, nprocess=1, nchunks=1, **soap_params)

it works. So I'll close this for now.

Hi, I just tried to run on my set up and it seems the problems comes from the threads and processes at the same time. So for the moment you can try to run with one of them set to 1 (threads I would recommend).
Thanks for reporting the bug.

Thank you for your reply.
Does this code support periodic structures (crystal structures) by the way?
Also, this is a minimal example in Jupyter notebook that I use to test performance:
%%time
import quippy
from GlobalSimilarity import get_globalKernel, get_environmentalKernels

from libmatch.chemical_kernel import Atoms2ChemicalKernelmat, deltaKernel

atoms = quippy.AtomsList('../4000.xyz', start=0, stop=10)

soap_params = {
    'atoms': atoms, 'centerweight': 1.0, 'gaussian_width': 0.5,
    'cutoff': 3.5, 'cutoff_transition_width': 0.5,
    'nmax': 8, 'lmax': 6, 'chemicalKernel': deltaKernel,
    'chemicalProjection': None,
    'chem_channels': True, 'nocenters': [1],'is_fast_average': False,
}

environmentalKernels = get_environmentalKernels(islow_memory=False, nthreads=1, nprocess=1, nchunks=1, **soap_params)

With just 10 crystal structures, I get:
nthreads=1, nprocess=1, nchunks=1 # => Wall time: 39.9 s
nthreads=2, nprocess=1, nchunks=1  #=> Wall time: 39.7 s
nthreads=1, nprocess=1, nchunks=2  #=> Freeze at 73%
nthreads=1, nprocess=2, nchunks=1  #=> Freeze at 0%
To summarize, the only thing that works is increasing nthreads, but it seems this does not actually speed up the calculations.
Another thing I noticed is that if I take only take 10 materials it takes 4 seconds per crystal structure. If I call get_environmentalKernels with 4000 materials it takes about 40 seconds per second crystal. I am not sure why it is slower, because get_environmentalKernels should just calculate the kernels for each material independently?
The crystal structures I am using have about 100 atoms in the unit cell on average. To process 4000 structures it would take about 46 hours to complete get_environmentalKernels at the current speed..

Thanks for your comments. Bear in mind that this implementation is a first attempt to make the whole procedure faster (compared to glosim) and I am well aware of its current limitations. I have a clear idea on how to improve it but I lack the time at the moment to reimplement all the possible settings.
I am surprised by your timings and that it takes so much time since I just computed the similarity between 7000 structures with about 50 atoms per cell in ~3hours. Maybe the docker container is not the best evironment to run heavy duty work.
Yes periodic boundary conditions are applied by default, so if you don't want them you should explicitly state it in the .xyz file as expected by the extended xyz format.
Let me summarize the action of these parameters to guide you:

Threads should help with large frames by splitting the computation.
Process trivially splits the computation of the soap vectors and if you have chunks also for the environmental similarity matrix. It should be very effective for the computation of the soap (low communications) while it is not so clear on the chunks (large overhead).
Chunks divides the computation of the similarity matrix in parts and each part can be handled by a different process. It is mainly intended for when the environmental similarity matrix does not fit in memory.

Thank you for your reply, that summary really helpful! It sounds like increasing nprocess would be most effective.
I think Docker should not be the reason for significant overhead. More importantly, with 1 process I am only using 1 core but I have 64 available, so if the soap vectors calculations are split trivially that should be able to cut from 46 hours to 43 minutes, that would be great! :smile:
The problem is that if I set for example nthreads=1, nprocess=8, nchunks=1, it just stalls at 0%:

I will try to see if I can debug why that is happening, because in the end I would like to scale up to 25,000 structures. Finally, the goal is to extract a train and test kernel from the complete matrix and do kernel ridge regression, roughly following the steps below (will expand into train/validation/test):
a = k + l * np.eye(len(Y))
coefficients = scipy.linalg.solve(a,Y, sym_pos=True,overwrite_a=True)
predictions = np.dot(k,coefficients)
print('MAE', np.mean(np.abs(Y - predictions)))

Another short question: In which scenario would setting nocenters=[1] make sense? I assume that it speeds up the calculations by skipping all the hydrogen and its positions should not be significant factors for most properties that people predict?

I found a work-around for the bug, by setting the following before running get_Soaps with nprocess greater than 1:
from tqdm import tqdm
tqdm.monitor_interval = 0

Source: https://github.com/tqdm/tqdm/issues/481
Now I can increase nprocess without stalling and it looks like multiple cores are used.

I have narrowed down the problem a bit: the program hangs if you use libmatch.soap.get_soap within a multiprocessing.Pool. If you do the same outside a Pool it just works, really weird.

Now that the libatoms website is online again I installed QUIPPY without docker and the issue of only being able to use one CPU is gone!
So this issue is now resolved.


Introspection removes inline comments

Bug description
prisma db pull removes inline comments from a schema.
Related https://github.com/prisma/prisma/issues/1370
How to reproduce

Create this schema.prisma file

datasource ds {
    provider = "sqlite"
    url      = "file:./file.db"
}

model User {
    id   String @id
    name String // Full name
}

Run yarn prisma db push.
Run yarn prisma db pull.
The comment "Full name" is removed.

Expected behavior
The comment should be preserved
Prisma information
NA
Environment & setup

Database: Tested with SQLite and Postgres

Prisma Version
prisma                  : 3.0.2
@prisma/client          : 3.0.2
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine 2452cc6313d52b8b9a96999ac0e974d0aedf88db (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli 2452cc6313d52b8b9a96999ac0e974d0aedf88db (at node_modules/@prisma/engines/migration-engine-darwin-arm64)
Introspection Engine    : introspection-core 2452cc6313d52b8b9a96999ac0e974d0aedf88db (at node_modules/@prisma/engines/introspection-engine-darwin-arm64)
Format Binary           : prisma-fmt 2452cc6313d52b8b9a96999ac0e974d0aedf88db (at node_modules/@prisma/engines/prisma-fmt-darwin-arm64)
Default Engines Hash    : 2452cc6313d52b8b9a96999ac0e974d0aedf88db
Studio                  : 0.423.0

Closing as duplicate of #1370


Rollback to 37762f7 egui_extras::layout::StripLayout::add()'s allocate_rect() strategy

At crates\egui_extras\src\layout.rs :
Allocate allocation_rect instead of max_rect.
Go back from this:
let response = self.ui.allocate_rect(max_rect, self.sense);
let response = response.with_new_rect(allocation_rect);
return (used_rect, response)

to this:
let response = self.ui.allocate_rect(allocation_rect, self.sense);
return (used_rect, response)

In order to allocate the
Closes https://github.com/emilk/egui/issues/3956.

The reverted code was introduced in:

https://github.com/emilk/egui/pull/3347

It seems like table row selection still works after reverting those lines, so 👍


Added missing logo reference to the ReactPHP topic

[X] I followed the contributing guidelines: https://github.com/github/explore/blob/master/CONTRIBUTING.md

I am:

[X] Suggesting edits to an existing Topic page

I'm suggesting these edits to an existing topic:

[X] Content (and my changes are in index.md)

Please explain why these changes are necessary:
When originally creating the PR adding the ReactPHP I wrongly assumed the added logo would be automatically picked up. This PR is to corrects that :blush: .

whoops! we should've caught that, too. (@mikemcquaid maybe an opp for more tests? 😇 )
thanks @WyriHaximus!

No problem @nayafia . And adding tests for this sounds like a good idea :+1:


Handle vuln_update notification post vulnerabilities refactor

Vulnerabilities refactor PR https://github.com/anchore/anchore-engine/issues/1001 disabled the result diffing in the vuln_update notification path
https://github.com/anchore/anchore-engine/blob/41615e56aedb9fd06288418e25733330776d7cf9/anchore_engine/services/catalog/catalog_impl.py#L1605-L1610
Update this logic to use the new vulnerability report format from policy-engine and implement the diff

@nightfurys can I close this issue or should it be left open as a tracker?

This is no longer needed. Was resolved in v0.10.0

The issue was fixed in https://github.com/anchore/anchore-engine/commit/05463ce09740cbb6451603a521add3835b57cf95


False Positive in Layout/EmptyLinesAroundArguments with inline private

With the following code fragment using an inline access specifier:
class Foo
  private def bar(baz)
    baz

    baz
  end
end

Expected behavior
The Layout/EmptyLinesAroundArguments cop should report no offense.
Actual behavior
Layout/EmptyLinesAroundArguments: Empty line detected around arguments.
Steps to reproduce the problem
Run rubocop on the fragment above.
The false positive will also be raised when using other inline access specifiers like protected, public, private_class_method, etc. Presence or absence of actual arguments on the method declaration does not matter -- the cop appears to be complaining about the blank line in the method body.  Removing that blank line stops the complaint.
RuboCop version
$ rubocop -V
0.52.1 (using Parser <IP_ADDRESS>, running on ruby 2.4.3 x86_64-darwin16)

Thanks for reporting this bug!  I'll take a crack at fixing this in the next few days.

After looking at this, there actually is not a bug here. private_class_method and other inline access modifiers are all methods of Module. So the offense getting thrown is because the method :bar is an argument of private(), and the arg contains an empty line.
We have a test that assures this kind of behavior - https://github.com/bbatsov/rubocop/blob/master/spec/rubocop/cop/layout/empty_lines_around_arguments_spec.rb#L41 .
It looks like this:
    it 'registers offense for empty line in-between one arg' do
      inspect_source(<<-RUBY.strip_indent)
        do_stuff([
          "some text",

          "some more text"
        ])
      RUBY
      expect(cop.messages)
        .to eq(['Empty line detected around arguments.'])
    end

This is understandable confusing. The example above is both an extra line around arguments, and an extra line inside an array literal. Similarly, the issue example is an extra line around arguments, and an extra line in a method definition.
Should the implementation & possibly cop name be changed to EmptyLinesBetweenArguments? -- I lean towards yes.


Could not unserialize key data (use .p8 file as private key file, use ES256 algorithm)

Such was the code:
APNS_AUTH_KEY = './APNsAuthKey_SP7C6832EZ.p8'
f = open(APNS_AUTH_KEY)
secret = f.read()
token = jwt.encode(
payload={
"iss": "8X2A38Q9VD",
"iat": time.time()
},
key= secret,
algorithm= 'ES256',
headers={
"alg": "ES256",
"kid": "SP7C6832EZ",
}
)
But it run with error：
Traceback (most recent call last):
File "Http2AuthToken.py", line 31, in
"kid": "SP7C6832EZ",
File "/usr/lib/python2.7/site-packages/jwt/api_jwt.py", line 56, in encode
json_payload, key, algorithm, headers, json_encoder
File "/usr/lib/python2.7/site-packages/jwt/api_jws.py", line 98, in encode
key = alg_obj.prepare_key(key)
File "/usr/lib/python2.7/site-packages/jwt/algorithms.py", line 226, in prepare_key
key = load_pem_private_key(key, password=None, backend=default_backend())
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/primitives/serialization.py", line 20, in load_pem_private_key
return backend.load_pem_private_key(data, password)
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/multibackend.py", line 303, in load_pem_private_key
return b.load_pem_private_key(data, password)
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1048, in load_pem_private_key
password,
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1217, in _load_key
self._handle_key_loading_error()
File "/usr/lib64/python2.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1289, in _handle_key_loading_error
raise ValueError("Could not unserialize key data.")
ValueError: Could not unserialize key data.
Is there somebody can explain?

Try updating your acme and boto libraries; I had a similar sort of error, and it worked for me.


regression in search filters

Affected module
UI
Describe the bug
In our previous versions, search aggregations/filters on left side are used to filter the content , each filter selection allows users to further drill down the documents.
However with 0.13.0 release we changed the filters to be post_filter which means it won't apply on the search aggregations.
Example

User selects snowflake as service

User still sees there are documents that are available in Tier and Tags. But they do not have any matching docs with snowflake as service

We should pass this as query_filter instead of post_filter
To Reproduce
Screenshots or steps to reproduce
Expected behavior
A clear and concise description of what you expected to happen.
Version:

OS: [e.g. iOS]
Python version:
OpenMetadata version: [e.g. 0.8]
OpenMetadata Ingestion package version: [e.g. openmetadata-ingestion[docker]==XYZ]

Additional context
Add any other context about the problem here.

[ ] Relation between two Facet filters will be -- AND
[ ] Should we persist filters across Tabs?
[ ] Should we override Facet Filter over Advance search & vice-versa?

yes
No, filters are specific to each entity , there might be some overlap, Tags, Tier but the rest will be different. An example service snowflake is a database service. They won't be any service like that in the rest of the entities
I don't understand. Can you explain a bit more

Fixed with https://github.com/open-metadata/OpenMetadata/pull/9214


Add support for Musl libc

Since Musl is sufficiently different from Glibc (see https://wiki.musl-libc.org/functional-differences-from-glibc.html), it requires a different import, which now should be applied to files that have import Glibc in them.

This feels like something swift-system should provide abstractions for tbh.

100%, but until then we have to import platform-specific libcs explicitly.

@swift-ci smoke test

@swift-ci test windows

100%, but until then we have to import platform-specific libcs explicitly.

This has already been pitched. Would you take that over, @MaxDesiatov, since you are adding Musl, including adding Bionic and the BSD libcs?
I'll be happy to help split off Bionic, I just don't want to push all this through the Evolution process.


Can't login with Firefox

Hello.
With the same user and password I can login with Chrome 91 browser but I can't do it with Firefox 89.
In Firefox I gets 404 error:

I had the same 404 problem with Firefox 94.0.2

was related to changes in AAD. Should be fixed in v3


Question: Create Control Charts?

I wonder is it possible to create Control Charts which displays the horizontal lines for 'Upper Control Limit', 'Lower Control Limit' along with 'Median' on a Line Chart?
Here is the link which demonstrates what I want to achieve.

Did you ever find a solution to this?

@wilpatterson I didn't give a try. Unfortunately, the project didn't get kick off. But, am still interested to figure out how the Control Charts can be plotted using this library.

Hello @coderabsolute 👋
Chartist has recently been updated to version 1.0 with some new exciting changes like ESM and TypeScript support. The migration guide can be found right here.
Please let me know if the question is still relevant to you so we can further look into it!


New config syntax: Matches

This is proposal of new matcher. Hoping to write or conditions at one match value with new type matches
- type: matches
  match: "info|debug"
  color: cyan
  target: word

Above case will color words on "info" or "debug".

match_either: if a line match a word either, then colorize all matched words or line
match_all: if a line match all words, then colorize all words or line

- type: match_either
  matches:
    - "info"
    - "debug"
  color: cyan
  target: word

Adding matches property for multiple matching conditions.


Get Host Address of the service

What version of docker are you running?

Did you build a custom version of registrator? If so, what is that image?
What is the exact command you are running registrator with?
What is the exact command you are running your container with?
A log capture of all the docker events before, during, and after the issue.
If relevant, Dockerfile for application that is having issues.

Description of the problem:
Get Host Address of the service running
When we do a
dig @<IP_ADDRESS>  -p 8600 exampe_server.service.consul. ANY , it returns the IP address of the docker container.

172.xx.x.x

Is there a way we can get the host that is running the service as well?

u can add -ip="host addr" after gliderlabs/registrator, and Before consul:///addr.


Add support for oh-my-zsh-style Git status

Adds support for oh-my-zsh's git status (the git status used in Spaceship < v3) (related to denysdovhan/spaceship-prompt 355). Requires the modular git setup in #1

Merged in 3e4c158


Splits the Lists table population in 500 items batches

Closes #2743
Summary
This PR splits the creation of rows in the Lists table into 500 item batches intead of a single bath with more than 5000 lines. This is to prevent a timeout issue during installation on some environments.

I did not test the syntax change but went for the minimal change that fixed the issue. I am not sure on support for the other syntax on various SQL server versions so wanted to avoid introducing that change in a bug fix release that is already in RC stage. SQL server script generation looks more like the current syntax except it generates a go in each line and has hardcoded values instead of variables.

Fair enough.  @ohine any thoughts

Just let me know if you want me to test other solutions on that same machine since I have a good testbed to reproduce. Make scare with the other solution was that it may not work everywhere (azure SQL, old versions of SQL server, etc.) Which I don't have a good testbed for.


add mapillary rules

@Raydiation I added this to the owncloud news app, but the scraper doesn't fetches the full content. What did I wrong?

Should actually work I think, maybe the content is generated with JavaScript, can you check?

The rulefile works for me with miniflux. Personally, I would remove the //h2[@class="blog-post__title"] part, since miniflux displays the heading twice.

@MorrisJobke Do you want to get this rule file merged or is this a (owncloud news app) support request?

Ah, did you turn on the website scraper for the website? You have to enable it by opting in, it's a separate button in the feed menu

@MorrisJobke Do you want to get this rule file merged or is this a (owncloud news app) support request?

Both :P I changed the indentation and it would be nice to have it upstream, isn't it? ;)
Feel free to merge it if it fits into your library.


Continue:  Google charts  and Google datatable

No idea why this happens in my system : https://github.com/Shekharrajak/daru-view/pull/35/commits/76aaeac458fed0ccc741ca35aa56ae600e80fff0#diff-a69a369d724b6660a77c91ad22ac4550L17

Merging it. I will  add specs in next PR.


Jump Search

Jump Search
Jump Search is an algorithm to find an element in a sorted array.
Java

I would like to work on this issue in Java.
@ricardoprins @VijayaGB98
Please assign it to me

Looking forward to see your contribution!

Solved by #296

@ricardoprins @VijayaGB98 I would like to work on this in C++.

It will be great to have your contribution on this!


Clean up comments

There are some old comments that need to be removed. Also, we can add some useful KDoc comments instead

 Clean up comments


Alert when trigger manual CAH during auto CAH

Travis did not passed due to yesterday AWS issues,
on restart the tests passed
https://travis-ci.com/github/maloel/librealsense/builds/175572224


Use className instead of class

React uses className.

I see. No wonder it didn't work. I think I accidentally used code from the original Bootstrap site instead of React-bootstrap...


button for taking to top of the page. For long trace list and long sp…

…an list traces.
We generally have traces with number of spans in range of 200-500. Page up button is useful for us. Hoping that it would be useful for others too. Attached screenshot shows this button on bottom right corner.
Thanks,
Nikhil.

sorry no-one's looked at this yet. I don't have an opinion.
@eirslett wdyt?

+1 Great work!