[SPARK-14549][ML] Copy the Vector and Matrix classes from mllib to ml in mllib-local

What changes were proposed in this pull request?
This task will copy the Vector and Matrix classes from mllib to ml package in mllib-local jar. The UDTs and since annotation in ml vector and matrix will be removed from now. UDTs will be achieved by #SPARK-14487, and since will be replaced by /* @ since 1.2.0 */
The BLAS implementation will be copied, and some of the test utilities will be copies as well.
Summary of changes:

In mllib-local/src/main/scala/org/apache/spark/ml/linalg/BLAS.scala

Copied from mllib/src/main/scala/org/apache/spark/mllib/linalg/BLAS.scala
logDebug("gemm: alpha is equal to 0 and beta is equal to 1. Returning C.") is removed in ml version.

In  mllib-local/src/main/scala/org/apache/spark/ml/linalg/Matrices.scala

Copied from mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala
@Since was removed, and we'll use standard /* @Since /* Java doc. Will be in another PR.
UDT related code was removed, and will use SPARK-13944 https://github.com/apache/spark/pull/12259  to replace the annotation.

In mllib-local/src/main/scala/org/apache/spark/ml/linalg/Vectors.scala

Copied from mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala
@Since was removed.
UDT related code was removed.
In def parseNumeric, it was throwing throw new SparkException(s"Cannot parse $other."), and now it's throwing throw new IllegalArgumentException(s"Cannot parse $other.")

In mllib/src/main/scala/org/apache/spark/mllib/linalg/Vectors.scala

For consistency with ML version of vector, def parseNumeric is now throwing throw new IllegalArgumentException(s"Cannot parse $other.")

mllib/src/main/scala/org/apache/spark/mllib/util/NumericParser.scala is moved to mllib-local/src/main/scala/org/apache/spark/ml/util/NumericParser.scala

All the throw new SparkException were replaced by throw new IllegalArgumentException

How was this patch tested?
unit tests

Test build #55571 has started for PR 12317 at commit 5795d54.

Test build #55571 has finished for PR 12317 at commit 5795d54.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
class DenseMatrix (
class SparseMatrix (
class DenseVector (val values: Array[Double]) extends Vector
class SparseVector (

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55571/
Test FAILed.

Test build #55574 has finished for PR 12317 at commit 1fba791.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55574/
Test FAILed.

Merged build finished. Test FAILed.

Test build #55589 has started for PR 12317 at commit f9473f5.

Test build #55589 has finished for PR 12317 at commit f9473f5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55589/
Test PASSed.

@dbtsai Please remove WIP from title when this is ready for review:)

Test build #55752 has started for PR 12317 at commit 69f6982.

Test build #55755 has started for PR 12317 at commit 38914b2.

Test build #55760 has started for PR 12317 at commit 45a0ee8.

Test build #55752 has finished for PR 12317 at commit 69f6982.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55752/
Test FAILed.

Merged build finished. Test FAILed.

Test build #55766 has started for PR 12317 at commit 150a60b.

Test build #55755 has finished for PR 12317 at commit 38914b2.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55755/
Test PASSed.

Test build #55760 has finished for PR 12317 at commit 45a0ee8.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55760/
Test PASSed.

Test build #55766 has finished for PR 12317 at commit 150a60b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55766/
Test PASSed.

Test build #55827 has started for PR 12317 at commit 35e2da5.

Test build #55827 has finished for PR 12317 at commit 35e2da5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55827/
Test PASSed.

@mengxr The versions of the dependencies have been moved to the parent POM.

+cc @srowen @MLnick

I'm making another pass.

Test build #55902 has started for PR 12317 at commit e626d1d.

Test build #55902 has finished for PR 12317 at commit e626d1d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/55902/
Test PASSed.

LGTM. Merged into master. Thanks!


fix inline nodes with nested marks

Inline nodes with nested marks result in markdown that might not be processed correctly (produces unexpected HTML output in Hugo).
Instead of _a[b](c)d_, the visual editor does _a_[_b_](c)_d_.
This fix ensures the former is always the case, even if the latter is received.

@erquhart If I write _a_[_b_](c)_d_andmoretext, then I get _a[b](c)__d_andmoretext.

@tech4him1 that seems like the intended output, isn't it!

@Benaiah There are two _ before d. I don't think that is intended, but I don't know the parser well enough -- whether it would be practical to change it or not.

@tech4him1 you're right, it should be grouping the "d" together with the previous text. Investigating.

Still planning on completing this.

Closing as stale.


Update Mediator

[ ] Modify it so that the receiver is before the sender in the arguments.
[x] Bug in broadcast method, when a participant has changed to falsy (undefined | null). Check that it is truthy before sending the message. Update docs to reflect this change (change delete this.participants[name] for this.participants[name] = null).

bug#2 fixed here: https://github.com/pgarciacamou/go-patterns/commit/6780041a85c0169b43ecaf5d22e1002d1488c5a5 7 days ago

this issue is being developed in #41


Improve Dockerfile to build complete app

Hi! I made some changes to the dockerfile, but I'm still running into a couple issues.
With this PR I'm aiming to deliver:

No host deps (besides docker) one-shot builds
Virtually instant incremental rebuilds
Minimize image size (at least minimized delta from ubuntu:20.04)
No setup on first run
User data exclusively contained in docker volume
App able to run with read-only root (except user data volume ofc)

Here's a list of changes:

Build rust frontend
Use a separate docker layer to build frontend deps; this reduces rebuild times when Cargo.toml is not changed to 0.2s (!)
Combine separate steps into one to reduce disk usage
Use 'multiple FROM' build strategy to reduce dependencies between layers which improves layer caching performance
Do haskell build (WIP)
Build app and static files in a build stage then copy the binaries to the final image to minimize image distribution size (WIP)
Begin to use /app for program files, use /data for data files backed by a docker VOLUME. This has both security and operational benefits. (WIP)
Add startup.sh script to do initalization and startup work (WIP)

There are still some things to do:

Haskell build does not do a separate "build deps" step a la rust. Ideally deps could be built first if possible, which would be cached for fast rebuilds
Haskell build is spread out between multiple dirs, so requires many COPY steps. Ideally all the haskell code would be in one dir
startup.sh does not yet initialize empty db on first run
/app doesn't have all required binaries
App doesn't use /data for storage yet

Specifically, I'm running into an error with libtorch. I suspect that you may know what's wrong by just looking at the error and dockerfile:
libtorch-ffi                     > Configuring libtorch-ffi-<IP_ADDRESS>...
libtorch-ffi                     > Cabal-simple_mPHDZzAJ_<IP_ADDRESS>_ghc-8.10.4: Missing dependencies on foreign
libtorch-ffi                     > libraries:
libtorch-ffi                     > * Missing (or bad) C libraries: c10, torch, torch_cpu
libtorch-ffi                     > This problem can usually be solved by installing the system packages that
libtorch-ffi                     > provide these libraries (you may need the "-dev" versions). If the libraries
libtorch-ffi                     > are already installed but in a non-standard location then you can use the
libtorch-ffi                     > flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
libtorch-ffi                     > the library files do exist, it may contain errors that are caught by the C
libtorch-ffi                     > compiler at the preprocessing stage. In this case you can re-run configure
libtorch-ffi                     > with the verbosity flag -v3 to see the error messages.
libtorch-ffi                     >

@junjihashimoto That helped and it builds successfully with docker build . -t openmemex:v1 now, thanks!
Now I'm running into an issue during startup. When I run docker run --rm openmemex:v1 the startup script runs stack run which shows an error: No executables found..
Should we use stack at all here? I.e. just copy over the binaries and execute them directly? Anyway, I suspect that I'm not copying the right files over from the build stage. Here is the complete directory listing as output on line 64 RUN cd /app && find .:
[4/4] STEP 4/9: COPY --from=build-haskell /src/.stack-work /app/.stack-work
--> Using cache 04cbfd39a9f556fcf04d6403b1326ae307e1202e62507d52723c6e573f5e4314
--> 04cbfd39a9f
[4/4] STEP 5/9: ADD ./openmemex.cabal ./startup.sh /app
--> b72845a73ff
[4/4] STEP 6/9: RUN cd /app; find
.
./frontend
./openmemex.cabal
./startup.sh
./.stack-work
./.stack-work/dist
./.stack-work/dist/x86_64-linux-tinfo6
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/autogen
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/autogen/Paths_openmemex.hs
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/autogen/cabal_macros.h
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/CrawlTools.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/CrawlTools.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/DB.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/DB.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/Date.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/Date.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/Main.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/Main.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/OCR.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/OCR.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/SQL.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/crawler/crawler-tmp/SQL.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/autogen
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/autogen/Paths_openmemex.hs
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/autogen/cabal_macros.h
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Backend.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Backend.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/DB.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/DB.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Date.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Date.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Main.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Main.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/OCR.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/OCR.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Parser.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/Parser.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/SQL.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/omx/omx-tmp/SQL.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/autogen
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/autogen/Paths_openmemex.hs
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/autogen/cabal_macros.h
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/API.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/API.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/CrawlTools.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/CrawlTools.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/DB.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/DB.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Date.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Date.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Main.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Main.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Models.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/Models.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/OCR.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/OCR.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/SQL.hi
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build/server/server-tmp/SQL.o
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/build-lock
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/package.conf.inplace
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/package.conf.inplace/package.cache
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/package.conf.inplace/package.cache.lock
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/setup-config
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/stack-build-caches
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/stack-build-caches/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/stack-build-caches/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/exe-server
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/stack-cabal-mod
./.stack-work/dist/x86_64-linux-tinfo6/Cabal-<IP_ADDRESS>/stack-setup-config-mod
./.stack-work/install
./.stack-work/install/x86_64-linux-tinfo6
./.stack-work/install/x86_64-linux-tinfo6/459704011977c3d53a5c7b28ec59b03237743612014b5604844c4eb393e89b46
./.stack-work/install/x86_64-linux-tinfo6/459704011977c3d53a5c7b28ec59b03237743612014b5604844c4eb393e89b46/8.10.4
./.stack-work/install/x86_64-linux-tinfo6/459704011977c3d53a5c7b28ec59b03237743612014b5604844c4eb393e89b46/8.10.4/pkgdb
./.stack-work/install/x86_64-linux-tinfo6/459704011977c3d53a5c7b28ec59b03237743612014b5604844c4eb393e89b46/8.10.4/pkgdb/package.cache
./.stack-work/install/x86_64-linux-tinfo6/459704011977c3d53a5c7b28ec59b03237743612014b5604844c4eb393e89b46/8.10.4/pkgdb/package.cache.lock
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/bin
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/bin/crawler
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/bin/omx
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/bin/server
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/doc
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/doc/openmemex-<IP_ADDRESS>
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/doc/openmemex-<IP_ADDRESS>/LICENSE
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/installed-packages
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/installed-packages/openmemex-<IP_ADDRESS>
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/pkgdb
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/pkgdb/package.cache
./.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/pkgdb/package.cache.lock
./.stack-work/stack.sqlite3
./.stack-work/stack.sqlite3.pantry-write-lock
./static
./static/.gitignore
./static/glider
./static/glider/LICENSE.txt
./static/glider/README.md
./static/glider/glider-compat.min.js
./static/glider/glider.css
./static/glider/glider.js
./static/glider/glider.min.css
./static/glider/glider.min.js
./static/glider/index.html
./static/glider/package.json
./static/index.html
./static/package.json
./static/pixitest.glsl
./static/pixitest.html
./static/pixitest.js
./static/screenshots
./static/static
./static/style.css
./static/thumbnails
./static/wasm.d.ts
./static/wasm.js
./static/wasm_bg.wasm
./static/wasm_bg.wasm.d.ts

I added the stack run server and rebased onto the new master/pr #5. I also edited the original PR post above to scope down some of the requirements that can be future enhancements. Now I'm running into this issue when starting the image:
[joe@fedora openmemex]$ docker run --rm openmemex:v1
+ cd /app
+ stack run server
/app/.stack-work/install/x86_64-linux-tinfo6/fd2670faba7dbbf9e1e40a7737ced7e4c833d0d730be4118a8ec0600db06f7e2/8.10.4/bin/server: error while loading shared libraries: libtokenizers_haskell.so: cannot open shared object file: No such file or directory

I've verified that /app/deps/tokenizers/libtokenizers_haskell.so exists.

Ok that issue was fixed by setting LD_LIBRARY_PATH, as present in the setenv file.

Can you try with the latest master commit? Made some tweaks to the server to initialize the database and create an initial entry.

Thanks!!


[WFU] Only load 1 page at a time.

Turns out the onEndReached detection was set on the wrong view.
Part of https://github.com/artsy/collector-experience/issues/698

There are a couple failing snapshots but at first glance it seems like they'd be fixed with a -u. Looks good to merge on 🍏 !

Woops! Will merge on green.


[extractor/tiktok] TikTokBaseIE update api hostname

IMPORTANT: PRs without the template will be CLOSED
Description of your pull request and other information

Modified the api hostname for TikTok. Looks like TikTok only shut down or changed the previous API hostname and did not start requiring additional signatures.
Cannot run tests for some reason due to a RecursionError :), so I manually ran some of the tests.
Fixes #5688
Template

Before submitting a pull request make sure you have:

[x] At least skimmed through contributing guidelines including yt-dlp coding conventions
[x] Searched the bugtracker for similar pull requests
[x] Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check one of the following options:

[x] I am the original author of this code and I am willing to release it under Unlicense
[ ] I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

[x] Fix or improvement to an extractor (Make sure to add/update tests)
[ ] New extractor (Piracy websites will not be accepted)
[ ] Core bug fix/improvement
[ ] New feature (It is strongly recommended to open an issue first)

Cannot run tests for some reason due to a RecursionError :),

Pls explain

Cannot run tests for some reason due to a RecursionError :),

Pls explain

test/test_download.py throws the following error for any extractor:
WARNING: Falling back to normal extractor since lazy extractor CrunchyrollIE does not have attribute real_class; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
Traceback (most recent call last):
  File "F:\GitHub\yt-dlp\test\test_download.py", line 269, in <module>
    inject_tests(normal_test_cases)
  File "F:\GitHub\yt-dlp\test\test_download.py", line 258, in inject_tests
    for test_case in test_cases:
  File "F:\GitHub\yt-dlp\test\helper.py", line 91, in gettestcases
    for ie in yt_dlp.extractor.gen_extractors():
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\__init__.py", line 20, in gen_extractors
    return [klass() for klass in gen_extractor_classes()]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\__init__.py", line 20, in <listcomp>
    return [klass() for klass in gen_extractor_classes()]
            ^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\lazy_extractors.py", line 36, in __new__
    instance = cls.real_class.__new__(cls.real_class)
               ^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\lazy_extractors.py", line 25, in __getattr__
    return getattr(cls.real_class, name)
                   ^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\lazy_extractors.py", line 25, in __getattr__
    return getattr(cls.real_class, name)
                   ^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\lazy_extractors.py", line 25, in __getattr__
    return getattr(cls.real_class, name)
                   ^^^^^^^^^^^^^^
  [Previous line repeated 985 more times]
  File "F:\GitHub\yt-dlp\yt_dlp\utils.py", line 5881, in __get__
    return self.func(cls)
           ^^^^^^^^^^^^^^
  File "F:\GitHub\yt-dlp\yt_dlp\extractor\lazy_extractors.py", line 32, in real_class
    cls._real_class = getattr(importlib.import_module(cls._module), cls.__name__)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Ben\AppData\Local\Programs\Python\Python311\Lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1203, in _gcd_import
RecursionError: maximum recursion depth exceeded

I'm guessing it's something on my end because I do not see other issues for this and recently reinstalled Windows and swapped files around.

I pulled video urls from the tests in tiktok.py and from the issue to test this change- works on my device

TikTokIE tests pass for me.

 Rebuild/remove it

It seems still not work.
url=https://api16-normal-c-useast1a.tiktokv.com/aweme/v1/feed/?aweme_id=7121336502398782747&version_name=26.1.3&version_code=260103&build_number=26.1.3&manifest_version_code=260103&update_version_code=260103&openudid=39e7cfc7f9023de1&uuid=1450823614373125&_rticket=1669974438356&ts=1669974438&device_brand=Google&device_type=Pixel+4&device_platform=android&resolution=1080*1920&dpi=420&os_version=10&os_api=29&carrier_region=US&sys_region=US&region=US&app_name=trill&app_language=en&language=en&timezone_name=America%2FNew_York&timezone_offset=-14400&channel=googleplay&ac=wifi&mcc_mnc=310260&is_my_cn=0&aid=1180&ssmix=a&as=a1qwert123&cp=cbfhckdckkde1
headers={'User-Agent': 'com.ss.android.ugc.trill/260103 (Linux; U; Android 10; en_US; Pixel 4; Build/QQ3A.200805.001; Cronet/58.0.2991.0)', 'Accept': 'application/json', 'Accept-Encoding': 'gzip, deflate'}, cookies={'odin_tt': '6f1dcfd0f0eb63399678b4fe18bf40b3be311bb0e1891a09df06fe65d4419662cc40351a8d7f918487930ab3152dc291809846f9f07e8c1bda91febe2059b42a127f38f63e1bc5b4fbeff1452a6051fc'}
response status=200, text=, content=b''

@guodong2019 can you show a verbose yt-dlp log of it not working? and what region are located in, if you don't mind sharing?

sorry, it worked now. I just called the aweme/detail/ endpoint.


Add data plane adoption job to install_yamls gate

Needs the Depends-On: to ensure we consume the content provider
openstack-operator image.

https://issues.redhat.com/browse/OSP-29169

Depends-On: https://review.rdoproject.org/r/c/rdo-jobs/+/50465

we need https://review.rdoproject.org/r/c/rdo-jobs/+/50465 to make sure that the job which runs here will consume the openstack-operator image built by the content provider.

recheck
needs a re-run now that https://review.rdoproject.org/r/c/rdo-jobs/+/50465 merged

recheck
TASK [mariadb_copy : MariaDB copy pre checks] **********************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": "#!/bin/bash\nset -euxo pipefail\n\n\nMARIADB_IMAGE=quay.io/podified-antelope-centos9/openstack-mariadb:current-podified\n\nPODIFIED_MARIADB_IP=<IP_ADDRESS>\nPODIFIE

https://logserver.rdoproject.org/71/571/74a4ddc44441cdc9264a5554740a4204dc3c849e/github-check/data-plane-adoption-github-rdo-centos-9-crc-single-node/7a9987d/controller/data-plane-adoption-tests-repo/data-plane-adoption/tests/logs/test_with_ceph_out_2023-10-16T05:04:42EDT.log

/approve

thanks for looking here and voting @fao89 . Still something further needed as the job is not using the built openstack-operator yet (we merged https://review.rdoproject.org/r/c/rdo-jobs/+/50465, but there is something further needed I'll work on that today).
Patch will stay in Draft until ready and I'll rebase on latest main when it is.

OK moving this to ready for review, last run shows we are consuming the built openstack-operator image:
https://logserver.rdoproject.org/71/571/cf14076f848d8473675c492d4ee8d0a276290323/github-check/data-plane-adoption-github-rdo-centos-9-crc-single-node/374e9d5/job-output.txt
2023-10-17 05:27:52.651322 | controller | TASK [Run make openstack using install_yamls_makes and pass content provider openstack-operator image name=install_yamls_makes, tasks_from=make_openstack] ***
2023-10-17 05:27:52.716827 | controller |
2023-10-17 05:27:52.716898 | controller | TASK [install_yamls_makes : Debug make_openstack_env var=make_openstack_env] ***
2023-10-17 05:27:52.764463 | controller | ok: [localhost] => {
2023-10-17 05:27:52.764544 | controller |     "make_openstack_env": {
2023-10-17 05:27:52.764560 | controller |         "OPENSTACK_IMG": "<IP_ADDRESS>:5001/openstack-k8s-operators/openstack-operator-index:f35f8a5576ab47f2bb05a9ddaf539b5264337c81",
2023-10-17 05:27:52.764567 | controller |         "OPENSTACK_REPO": "~/src/github.com/openstack-k8s-operators/openstack-operator"
2023-10-17 05:27:52.764573 | controller |     }
2023-10-17 05:27:52.764579 | controller | }

and checking the pod debug logs there: https://logserver.rdoproject.org/71/571/cf14076f848d8473675c492d4ee8d0a276290323/github-check/data-plane-adoption-github-rdo-centos-9-crc-single-node/374e9d5/controller/pod/openstack-operator-index-49f9d-describe.txt
     Image:          <IP_ADDRESS>:5001/openstack-k8s-operators/openstack-operator-index:f35f8a5576ab47f2bb05a9ddaf539b5264337c81

:+1:


[Docker] No hosts or host table rows not found

Hi,
if I run noip-renew from shell everything works fine.
Running on docker I got "No hosts or host table rows not found"
I followed these steps:
export my_username='myusername'
export my_password='mypassword'
export my_host_num='1'
export debug_lvl=2
docker build -t loblab/selenium:debian .

and then (not considering error about No such file or directory: '/usr/local/bin/noip-renew-skd.sh' at the end of run) I've got this...
$  docker run --network host loblab/selenium:debian ${my_username} ${my_password} ${my_host_num} ${debug_lvl}
[2021/02/17 22:57:10] - Debug level: 1
[2021/02/17 22:57:10] - Opening https://www.noip.com/login...
[2021/02/17 22:57:12] - Logging in...
[2021/02/17 22:57:15] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/02/17 22:57:18] - No hosts or host table rows not found
Traceback (most recent call last):
  File "/home/loblab/noip-renew.py", line 167, in run
    if not self.update_hosts():
  File "/home/loblab/noip-renew.py", line 90, in update_hosts
    hosts = self.get_hosts()
  File "/home/loblab/noip-renew.py", line 159, in get_hosts
    raise Exception("No hosts or host table rows not found")
Exception: No hosts or host table rows not found

my_host_num variable was taken as debug level
No hosts or host table rows not found

Same error here. What's the solution ?

@snakuzzo @RomyxBaps Sometimes there will be pop up boxes shown and the script will be failed. You might want to check the debug1.png, debug2.png and exception.png that the script produced. You could output the images by,
# list out all closed containers to find the container id
docker ps -a

# export the files assuming the container id is abc123
docker cp abc123:/home/loblab/debug1.png .
docker cp abc123:/home/loblab/debug2.png .
docker cp abc123:/home/loblab/exception.png .

The problem will show in debug2.png, as that is the screenshot taken in the second step.

I think popup is not the problem.
As I said before...it works fine outside docker.
It doesn't work inside docker...
Anyway...@michaeleekk I tried...but I cannot copy files when container is not running...and container exit too quickly. So... I created the container mounting a volume
docker run --network host -v /home/snakuzzo/noip_renew_dir:/home/lolab loblab/selenium:debian ${noip_username} ${noip_password}
but no files are written...
snakuzzo@SnakuzzoWork:~/noip_renew_dir$ pwd
/home/snakuzzo/noip_renew_dir
snakuzzo@SnakuzzoWork:~/noip_renew_dir$ docker run --network host -v /home/snakuzzo/noip_renew_dir:/home/lolab loblab/selenium:debian ${noip_username} ${noip_password}
/usr/local/bin/noip-renew-skd.sh: line 9: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 9: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 13: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 13: sudo: command not found
[2021/03/13 18:04:22] - Debug level: 1
[2021/03/13 18:04:22] - Opening https://www.noip.com/login...
[2021/03/13 18:04:25] - Logging in...
[2021/03/13 18:04:28] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/03/13 18:04:31] - No hosts or host table rows not found
snakuzzo@SnakuzzoWork:~/noip_renew_dir$ ls -lart
total 8
drwxr-xr-x 122 snakuzzo snakuzzo 4096 mar 13 19:03 ..
drwxrwxr-x   2 snakuzzo snakuzzo 4096 mar 13 19:03 .

@snakuzzo can you try docker container prune and rerun the docker command again ? I once had the same problem as you and cleaning up the finished containers solved my problems.
And for your info, if you didn’t add --rm in your docker run command, the container hasn’t been cleaned up yet and could be found with docker ps -a. If you know the exact path of your files, you could copy the files out of that finished container with docker cp.

Done, but same result...
snakuzzo@SnakuzzoWork:~$ docker container prune
WARNING! This will remove all stopped containers.
Are you sure you want to continue? [y/N] y
Deleted Containers:
7e3bcda528c7fb1460ad6fc53be6af1f0873448c14775ba306787b9434c4fe83
c3e7579ad83db27856c5b18f1663777e95911444a0c48d7d5b81cf00e759d6a4
63059f6af38add0917cc3d1a1e0b84de614755af777307724b844f6558faf058

Total reclaimed space: 422.7kB
snakuzzo@SnakuzzoWork:~$ docker container ls -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES
snakuzzo@SnakuzzoWork:~$ docker run --network host loblab/selenium:debian ${noip_username} ${noip_password}
/usr/local/bin/noip-renew-skd.sh: line 9: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 9: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 13: sudo: command not found
/usr/local/bin/noip-renew-skd.sh: line 13: sudo: command not found
[2021/03/13 23:12:04] - Debug level: 1
[2021/03/13 23:12:04] - Opening https://www.noip.com/login...
[2021/03/13 23:12:36] - Logging in...
[2021/03/13 23:12:36] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/03/13 23:12:40] - No hosts or host table rows not found
snakuzzo@SnakuzzoWork:~$ docker container ls -a
CONTAINER ID   IMAGE                    COMMAND                  CREATED          STATUS                     PORTS     NAMES
a266a1d28b87   loblab/selenium:debian   "python3 /home/lobla…"   46 seconds ago   Exited (2) 7 seconds ago             determined_dubinsky
snakuzzo@SnakuzzoWork:~$ docker cp a266a1d28b87:/home/loblab/debug1.png .
Error: No such container:path: a266a1d28b87:/home/loblab/debug1.png

@snakuzzo hmm... I’m not totally sure then...
Maybe try any of these,

Compare the docker files with the one you have and the one on the repo
pull the source again, rebuild the image and try again
Try to find out the differences between the one you run on the shell and the docker version to see if there are different settings

Hope you get it works in the end.

Same error and impossible to find png.

@jellyfinuser If you are using docker, you will need to either mount the folder or use docker cp command, I mentioned above, to copy the images out of the container.

@michaeleekk ...
snakuzzo@SnakuzzoWork:~/prova$ docker run --network host loblab/selenium:debian ${noip_username} ${noip_password}
[2021/03/26 22:26:54] - Debug level: 1
[2021/03/26 22:26:54] - Opening https://www.noip.com/login...
[2021/03/26 22:26:56] - Logging in...
[2021/03/26 22:26:56] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/03/26 22:27:00] - No hosts or host table rows not found
Traceback (most recent call last):
  File "/home/loblab/noip-renew.py", line 167, in run
    if not self.update_hosts():
  File "/home/loblab/noip-renew.py", line 90, in update_hosts
    hosts = self.get_hosts()
  File "/home/loblab/noip-renew.py", line 159, in get_hosts
    raise Exception("No hosts or host table rows not found")
Exception: No hosts or host table rows not found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/loblab/noip-renew.py", line 200, in <module>
    sys.exit(main())
  File "/home/loblab/noip-renew.py", line 181, in main
    return (Robot(noip_username, noip_password, debug)).run()
  File "/home/loblab/noip-renew.py", line 172, in run
    subprocess.call(['/usr/local/bin/noip-renew-skd.sh', "*", "*", "False"])
  File "/usr/lib/python3.7/subprocess.py", line 323, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/bin/noip-renew-skd.sh': '/usr/local/bin/noip-renew-skd.sh'
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$ docker container ls -a
CONTAINER ID   IMAGE                    COMMAND                  CREATED          STATUS                     PORTS     NAMES
4012beb9db2b   loblab/selenium:debian   "python3 /home/lobla…"   13 seconds ago   Exited (1) 4 seconds ago             amazing_rosalind
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$
snakuzzo@SnakuzzoWork:~/prova$ docker cp amazing_rosalind:/home/loblab/debug1.png .
Error: No such container:path: amazing_rosalind:/home/loblab/debug1.png

I have the same problem and i managed to open the debug2.png

Hi @giorgis1987 can you explain how ro get debug images?

Ok...thanks.
In previous runs I used username, but now to login it needs email address.
Fixed this and...this is the result of my last try...
docker run --network host loblab/selenium:debian ${noip_username} ${noip_password}
[2021/03/30 17:21:02] - Debug level: 1
[2021/03/30 17:21:02] - Opening https://www.noip.com/login...
[2021/03/30 17:21:04] - Logging in...
[2021/03/30 17:21:05] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/03/30 17:21:08] - No hosts or host table rows not found

...I got the image exception.png

Somthing goes wrong.
I tried also using script outside docker...and it worked...
[2021/03/30 19:28:54] - Debug level: 2
[2021/03/30 19:28:54] - Opening https://www.noip.com/login...
[2021/03/30 19:28:57] - Logging in...
[2021/03/30 19:29:01] - Opening https://my.noip.com/#!/dynamic-dns...
[2021/03/30 19:29:03] - xxxxxxxxxxxxxxxxxx expires in 28 days
[2021/03/30 19:29:03] - Confirmed hosts: 0

I tried again...now it works with username.


Why negative gain exists ?

I think all the gain in one tree should be non-negative because if it's negative, the xgboost would not split it. But when I run get_score by each tree in a model, I find some gain may be negative. How should I interpret this case?
like the following picture, when I get the gain in first tree, the gain for f37 is 0.28, but when I get the gain for f37 in the first 100 trees, it becomes 0.135

The gain is normalized by the number of splits created by this feature. To prevent the normalization, use total_gain instead.

Does the number of splits means the value I get from get_score(importance_type==weight) ?

yes.


Cannot add more than one click point to point layer

Description:
I want to add click points and then use those click points to update segment objects in segment anything model SAM.
However, after selecting Add points with mouse clicks from Add points layer dialog, I could add only one point.
Any followed clicks did not add any points, even if I open Add points layer dialog and add another colored point type.
Way to reproduce:
Select add points with mouse clicks in Add points layer dialog.

After that, clicking anywhere, only the first click produces a point. Any later clicks did not show anything.
(Optional) Link to data to reproduce error:
No response
Terminal output:
20-02-2024, 08:48:18 - cellacdc-logger-acdc_gui - INFO:
------------------------
Initializing GUI v1.4.21
------------------------

20-02-2024, 08:48:18 - cellacdc-logger-acdc_gui - INFO:
------------------------
Initiliazing profilers...
------------------------

20-02-2024, 08:48:18 - cellacdc-logger-acdc_gui - INFO:
------------------------
GUI ready.
------------------------

20-02-2024, 08:49:49 - cellacdc-logger-acdc_gui - INFO:
------------------------
Asking to select a folder path...
------------------------

20-02-2024, 08:49:51 - cellacdc-logger-acdc_gui - INFO:
------------------------
Loading from D:/tmp/grayscale/20240220_082140_acdc/Images
------------------------

20-02-2024, 08:49:51 - cellacdc-logger-acdc_gui - INFO:
------------------------
Using .tif file (D:/tmp/grayscale/20240220_082140_acdc/Images\9Z15003-1_crop.tif)...
------------------------

20-02-2024, 08:49:51 - cellacdc-logger-acdc_gui - INFO:
------------------------
Reading  channel metadata...
------------------------

20-02-2024, 08:49:51 - cellacdc-logger-acdc_gui - INFO:
------------------------
Lazy loader paused.
------------------------

20-02-2024, 08:49:51 - cellacdc-logger-acdc_gui - INFO:
------------------------
Closing lazy loader...
------------------------

20-02-2024, 08:49:52 - cellacdc-logger-acdc_gui - INFO:
------------------------
Segmentation filename: D:/tmp/grayscale/20240220_082140_acdc/Images\9Z15003-1_crop_segm.npz
------------------------

20-02-2024, 08:49:54 - cellacdc-logger-acdc_gui - INFO:
------------------------
Loading 20240220_082140_acdc\Images\9Z15003-1_crop.tif...
------------------------

20-02-2024, 08:49:54 - cellacdc-logger-acdc_gui - INFO:
------------------------
Loaded paths:
Segmentation file name: 9Z15003-1_crop_segm.npz
ACDC output file name 9Z15003-1_crop__acdc_output.csv
------------------------

20-02-2024, 08:49:55 - cellacdc-logger-acdc_gui - INFO:
------------------------
Full dataset shape = Not Loaded
------------------------

20-02-2024, 08:49:55 - cellacdc-logger-acdc_gui - INFO:
------------------------
Loaded dataset shape = (1, 424, 567)
------------------------

20-02-2024, 08:49:55 - cellacdc-logger-acdc_gui - INFO:
------------------------
Number of frames = 1
------------------------

20-02-2024, 08:49:55 - cellacdc-logger-acdc_gui - INFO:
------------------------
Number of z-slices per frame = 1
------------------------

20-02-2024, 08:49:55 - cellacdc-logger-acdc_gui - INFO:
------------------------
Creating graphical items...
------------------------

20-02-2024, 08:49:56 - cellacdc-logger-acdc_gui - INFO:
------------------------
Initializing measurements...
------------------------

20-02-2024, 08:49:56 - cellacdc-logger-acdc_gui - INFO:
------------------------
Setting GUI mode to "Snapshots"...
------------------------

20-02-2024, 08:49:56 - cellacdc-logger-acdc_gui - INFO:
------------------------
20240220_082140_acdc || Basename: 9Z15003-1_crop || Segmented channel:  || Segmentation file name: _segm.npz
------------------------

20-02-2024, 08:49:56 - cellacdc-logger-acdc_gui - INFO:
------------------------
Autosaving worker started.
------------------------

20-02-2024, 08:50:13 - cellacdc-logger-acdc_gui - INFO:
------------------------
[WARNING]: ID 1 does not exist (add points by clicking)
------------------------

20-02-2024, 08:50:18 - cellacdc-logger-acdc_gui - INFO:
------------------------
[WARNING]: ID 1 does not exist (add points by clicking)
------------------------

20-02-2024, 08:50:37 - cellacdc-logger-acdc_gui - INFO:
------------------------
Closing autosaving worker...
------------------------

20-02-2024, 08:50:37 - cellacdc-logger-acdc_gui - INFO:
------------------------
Autosaving worker closed.
------------------------

20-02-2024, 08:50:38 - cellacdc-logger-acdc_gui - INFO:
------------------------
Clearing memory...
------------------------

20-02-2024, 08:50:38 - cellacdc-logger-acdc_gui - INFO:
------------------------
Closing GUI logger...
------------------------

Version information:
(acdc) D:\tmp>python -c "import sys; print(sys.version); import platform; print(platform.platform()); import cellacdc; print(f'Cell-ACDC version: {cellacdc.__version__}')"
3.9.18 | packaged by conda-forge | (main, Dec 23 2023, 16:29:04) [MSC v.1929 64 bit (AMD64)]
Windows-10-10.0.19045-SP0
Cell-ACDC version: 1.4.21

Hi @ptlzon, sorry about this one, it was a bug in adding points in 2D data.
I fixed it now and I released v1.4.23. To upgrade activate the acdc environment and run python -m pip install --upgrade cellacdc.
Since you mentioned that you want to use SAM with input points I added that feature too (in previous versions it was possible to run SAM only in automatic mode without prompts).
The idea is to first click on points with the Add points by clicking function that you already tried with the difference that you will also need to be careful with the id of the points. If you click the mouse's left button you will add a new object per point. If you want to add more points for the same object, click with the right button. You will see the id of the point just beside the actual symbol of the point.
Next, you save these points to a file with the save button. Finally, you will be able to input the end-name of the table to SAM as a parameter like in this screenshot:

Let me know if it works, thanks!

Hi @ElpadoCan
Thanks for the quick fix. SAM works as you describe now.
However, I am not quite sure how to input both positive points and negative points to SAM.
I created two point layers (using only right clicks so as not to create additional IDs) and I expect them to be positive and negative prompts for SAM.
However, when I fed the point click list to SAM like above, the results seemed to only consider points layer number two.
Should I create another issue.
If the above is not clear, I will provide more details with example screenshot later. I'm sorry, this week I'm occupied.
Your dedication to improving this FREE tool is awesome. Thanks a lot.
Lam

Hi @ptlzon,

Your dedication to improving this FREE tool is awesome. Thanks a lot.

No problem at all, I'm happy if Cell-ACDC can be useful!

I created two point layers (using only right clicks so as not to create additional IDs) and I expect them to be positive and negative prompts for SAM.

Each point layer will result in a separate table and since you can provide a single table to SAM only one will be considered. You need to create a single points layer including negative points. To add negative points to the same layer you need to give them an id = 0. In the top toolbar, you will be able to manually enter the id of the points (see screenshot below). Enter 0 and then use the right-click to add negative points. As soon as you click with the left button the id will be increased by 1. So for example you could enter 0, right-click on negative points and once you are done you click the left button on the first positive point and then keep using right-click to add positive points with the same id.
TL;DR Negative points have id = 0.
In the future version, this will be documented better and I will also add the middle click (the scrolling wheel on Windows and Cmd+click on macOS) to add negative points. Let me know if it works, thanks!
Best,
Francesco

Hi @ElpadoCan
Thanks for the clarification.
I could use negative and positive points now with your instruction.
However, it seems that for my case, the combination of positive/negative points did not work as I expect (I was hoping it to produce much nicer output) or I need to spend more time understanding how to use it.
Thanks a lot.
Sincerely,
Lam

I'm glad to heart it "works". I guess how well it works it comes down to the dataset but if you see any suspicious behaviour feel free to report it.
Also, if you want to discuss dataset specific issues feel free to open a discussion on the image.sc forum https://forum.image.sc/
Cheers
Francesco


Add KO_EXTRA_ARGS

Changes
Add the ability to add ko extra args and move the existing arg --preserve-import-paths to be a default for the task.
This way the pipeline run can control settings and change the default path based on the registry, as well as control other ko settings.
Submitter Checklist
As the author of this PR, please check off the items in this checklist:

[x] Has Docs if any changes are user facing, including updates to minimum requirements e.g. Kubernetes version bumps
[x] Has Tests included if any functionality added or changed
[x] pre-commit Passed
[x] Follows the commit message standard
[x] Meets the Tekton contributor standards (including functionality, content, code)
[x] Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
[x] Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings). See some examples of good release notes.
[ ] Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes
NONE

/kind misc

/lgtm


handle windows paths in product definitions

Why:
resolves #316
What's being changed:
product URLs are being computed based on file system paths so need to be slashed for Windows
Check off the following:

[ ] All of the tests are passing.
[ ] I have reviewed my changes in staging.
[ ] For content changes, I have reviewed the localization checklist
[ ] For content changes, I have reviewed the Content style guide for GitHub Docs.

where's my daughter

link qr code is: line://au/q/o5e7CQe4aZp7tdeUzmFMRAXZG5lkLwqA

link qr code is: line://au/q/o5e7CQe4aZp7tdeUzmFMRAXZG5lkLwqA


cmd/go: What should the new semver tag be for +incompatible modules?

Summary
When a +incompatible module decides to introduce a go.mod file and updates all of its import paths to include vX, what should their new git tag be? Should they bump their patch, minor, or major?
What version of Go are you using (go version)?
1.11
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
All
What did you do?
Introduce go.mod to a +incompatible package: https://github.com/google/martian/pull/262
What did you expect to see?
A clear instruction on how to migrate such libraries.
What did you see instead?
I'm not sure if google/martian should go from v2.0.0 to v2.0.1 or v2.1.0 or v3.0.0

I briefly discussed this with Go team members at GopherCon but the question has come up a few times from library owners so I thought I'd create an issue for it.
cc: @bcmills

Closing as a dup of (the newly-reopened) #25967.


Todo brought over from the main repo

[ ] https://github.com/cr-api/cr-api/issues/237
[ ] https://github.com/cr-api/cr-api/issues/240

we got em issue movers


Minor fix in github workflow manifest for the path based frontend app build

Proposed changes
Minor fix in github workflow manifest for the path based frontend app build
Pipeline-
Types of changes
What types of changes does your code introduce to Litmus? Put an x in the boxes that apply

[ ] New feature (non-breaking change which adds functionality)
[ ] Bugfix (non-breaking change which fixes an issue)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Documentation Update (if none of the other choices applies)

Checklist
Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

[ ] I have read the CONTRIBUTING doc
[ ] I have signed the commit for DCO to be passed.
[ ] Lint and unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works (if appropriate)
[ ] I have added necessary documentation (if appropriate)

Dependency

Please add the links to the dependent PR need to be merged before this (if any).

Special notes for your reviewer:

Tested in this pipeline- https://github.com/rajdas98/litmus/actions/runs/1026294857


Permission Denied at step start

This is the output from my workflow at the very beginning:

/usr/bin/docker pull ghcr.io/trstringer/manual-approval:1.9.0
  permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.2[4](https://github.com/wasuaje/fastERPTodocaser/actions/runs/5647730094/job/15298594526#step:2:4)/images/create?fromImage=ghcr.io%2Ftrstringer%2Fmanual-approval&tag=1.9.0": dial unix /var/run/docker.sock: connect: permission denied

the code
    steps:
      - name: Manual Workflow Approval
        uses<EMAIL_ADDRESS>        with:
          secret: ${{ secrets.GITHUB_TOKEN }}
          approvers: wasuaje
          minimum-approvals: 1

any suggestion?
Update: I think the issue it is trying to run /usr/bin/docker, for security reasons github runner is in a non-proviledge user which only has SUDO docker access, so I usually make "sudo docker xxxx yyy zzz". This action points to "/usr/bin/docker". Any way to bypass this config?
I tried hard to make aliases, through .env file setting up a pre-run script, which seems to work (alias docker='sudo docker') At SO level works, and run setup shows it works, but somehow can't make it work with this action.
Any help is appreciated

I think that might be a GitHub Actions feature if I understand your issue correctly.


Create more formal map of climate zones

Create a more formal version of the climate zones map, likely through the use of GIS tools (e.g. a shapefile that contains the climate zones as polygons). The initial goal is a map figure with more refined outlines of each zone (using the current map figure as the starting point), while future work can focus on things like formal definitions of the zones and computing the zone given a lat-lon coordinate.

@wholmgren Note: I'm opening this issue here, but let me know if it's better to open an issue (and later a PR) against the website since that's where the current map figure exists.

Did you see #153? Perhaps a duplicate unless I'm misunderstanding a distinction.

Somewhat of a duplicate of #153, although I meant this issue to focus solely on the visualization portion (i.e. create a shapefile with refined outlines of the zones) as a first step.

Ah, ok. #153 should probably be moved to solarforecastarbiter-api and this one to either solarforecastarbiter_dashboard or solararbiter.github.io. @lboeman @alorenzo175 ?

Agreed. If we're not tracking any code for the production of the climate zone shapefile and just want a reminder/place for discussion, it should live in solararbiter.github.io. Unless we're trying to have some interactive visualization?

In the long run we would like to have a map of at least reference data sites (and possibly any site a user has access to) with a climate zone layer. Certainly a dashboard thing if including user sites. Probably also helpful to have a simple map of the climate zones that lives on the static page. Reference data plus climate zones could probably live in either repo.

Thanks for the feedback. I'll move this issue over to solararbiter.github.io to focus on adding a revised version of the (static) climate zone map to the website (although the results can probably be a starting point for future revisions, e.g., adding the climate zones onto an interactive map showing the reference data sites).


Add from feature button

Adds functionality to support adding a widget to the homepage from a mascot announcement

Contributor License Agreement adherence:

[X ] This Contribution is under the terms of Individual Contributor License Agreements (and also Corporate Contributor License Agreements to the extent applicable) appearing in the Apereo CLA roster.

Linting errors:
https://travis-ci.org/UW-Madison-DoIT/angularjs-portal/builds/275905557?utm_source=github_status&utm_medium=notification
https://travis-ci.org/UW-Madison-DoIT/angularjs-portal/builds/275905632?utm_source=github_status&utm_medium=notification
https://ci.appveyor.com/project/ChristianMurphy/angularjs-portal-npp5x/build/1.0.31
https://ci.appveyor.com/project/ChristianMurphy/angularjs-portal-npp5x/build/1.0.32


improvement tutorial page layout

improvement tutorial page layout by not wrapping tutorial title text
And also update hexo version

@aashish24
I kind of already pushed this change to the site.
But I can simulate a before and after:
Before:

After:


Drop Python3.8, Support Python3.12 and newer xclim/xarray/pandas

The only failing tests seem to be related to the changes in pandas frequency strings:
===================== short test summary info =====================
FAILED tests/test_forecasting.py::test_forecasting_GEPS[GR4JCN] - KeyError: 'h'
FAILED tests/test_commands.py::test_gridded_forcing - KeyError: 'h'
FAILED tests/test_coords.py::test_infer_scale_and_offset - KeyError: 'h'
FAILED tests/test_hindcasting.py::TestHindcasting::test_hindcasting_GEPS - KeyError: 'h'


Update dependencies for evidence datasource parsers

We've been notified that some of the dependencies in the evidence data source parsers are pose potential risks.
The following issues were found:

Python Charmers Future denial of service vulnerability High
GitPython vulnerable to Remote Code Execution due to improper user input validation High
ReDoS in py library when used with subversion Moderate

What's need to be done here:

[ ] Update packages used by the parser.
[ ] See if the updated versions are not posing any risk.

@DSuveges I'm just looking at recent PRs and trying to determine the status of this ticket, as far as I can see this hasn't been done yet? If so I can take back over and get this done

No, I didn't have the capacity to address this issue. I assume, other than an eyesore, they pose small risk. However, the sooner we address them is the better, but no direct pressure as we are done for the release.

Currently we have a number of import in the repo:

xml
urllib
time
tempfile
sys
shutil
requests
re
random
pyspark
pronto
pkg_resources
pathlib
pandas
os
optparse
numpy
math
logging
json
gzip
csv
argparse
typing
retry
psutil
pandarallel
ontoma
itertools
datetime

Most of them are built in Python libs so we are kind of alright. I would suggest dropping utils/get_phenodigm_score_distribution.py. This is the only script that imports csv and optparse. There's absolutely no need to work with them. Once these are dropped, the requirements can be re-defined, which I'm sure will solve the security issues.

@DSuveges The script you mentioned is a relic of the the long past, and we're neither using it nor installing dependencies for it. I'll remove it as part of this ticket.
The three modules flagged by dependabot (future, GitPython, py) are all probably transitive dependencies of other packages on which we depend directly. Anyway, I'll address that as well.

Yes, I didn't see it anywhere and wan't able to quickly track them down. I guess that's the price to pay for maintain a lockfile.


Run Recline from a CDN?

All the examples involve downloading Recline and uploading to a webserver, or running it locally. Is it possible to load the scripts from a CDN instead? Maybe directly from Github, or a public CDN like https://cdnjs.com/ ?
I see version 0.5 is available on http://www.jsdelivr.com/#!recline so maybe it just needs to be updated.

WONTFIX.
Reason: age of issue and spring cleaning (not anything about the idea!)


Should it be bigger?
Should we use the yellow version instead of this one?

"Cadet" should not appear anywhere as the name of the system. It should be "Source Academy". We should also not let things like "frontend" or "backend" spill over into the user namespace.

Actually that's pretty trivial so I'll go ahead and make the changes


Type Error: TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

Hi,
I tried to run the code by downloading the dataset of only Line Chart. I was able to make all the files. When I try to get the result using test_pipe_type_cloud.py, I get the below error -
loading parameters at iteration: 50000
Traceback (most recent call last):
File "test_pipe_type_cloud.py", line 240, in
methods = Pre_load_nets(args.type, 0, args.data_dir, args.cache_path)
File "test_pipe_type_cloud.py", line 82, in Pre_load_nets
db_line, nnet_line = load_net(50000, "CornerNetLine", data_dir, cache_dir,
File "test_pipe_type_cloud.py", line 57, in load_net
db = datasets[dataset](configs["db"], split)
File "C:\Users\RoyAnshu\AppData\Local\Programs\Python\Python38\Scripts\Master thesis\source code\DeepRule-master\db\coco.py", line 446, in init
cache_dir = system_configs.cache_dir
File "C:\Users\RoyAnshu\AppData\Local\Programs\Python\Python38\Scripts\Master thesis\source code\DeepRule-master\config.py", line 175, in cache_dir
if not os.path.exists(self._configs["cache_dir"]):
File "C:\Users\RoyAnshu\AppData\Local\Programs\Python\Python38\lib\genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
Any solution?

@soap117 could you please have a look into this. I am facing the same here. Thanks

Maybe you should try adding parameters to the --cache_path.

yeah. that was the reason. thanks. maybe this issue can be closed

It works for me as well.
I have now run into a driver issue. What version of CUDA toolkit and torch version are you using?
Will Cuda 9.1, 9.2 work?


Multiple Input detected in a loop

Does this affect the legacy HoloToolkit (master) or the Mixed Reality Toolkit (mrtk_release)?
Mixed Reality Toolkit VNext
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
https://github.com/achaperon/MRTK_Test
Steps to reproduce the behavior:

Open project and compile for HoloLens on IL2CPP
Launch the App
Make Airtap
Show the marker in front of the HoloLens and make Airtap at the same time

Expected behavior
Only one airtap should be detected per gesture
Actual behavior
Sometimes, just showing the marker trigger OnInputDown with the "select" Input Action
When the marker is in front of the HoloLens and you make the "reday" gesture. Multiple Airtap are detected (1 per update).
Actually, at each Update we have OnInputDown and OnInputUp with the "select" Input Action.
Unity Editor Version
2018.2.10
Mixed Reality Toolkit Release Version
Beta Release 2018.9 MRTK VNext
Additional context
Add any other context about the problem here.

Thanks for the report

Should be fixed with #3293. Please report back and reopen if not.


Problem with registration of some LwM2M clients after upgrading to leshan-2.0.0-M7

We noticed that after upgrading to leshan-2.0.0-M7, some devices stopped successfully registering in leshan server. We are observing following error:
[DEBUG] Invalid request [CON-POST   MID=50791, Token=150001, OptionSet={"Uri-Path":"rd", "Content-Format":"application/link-format", "Uri-Query":["lt=86400","lwm2m=1.1","ep=leshan100"]}, "</1/0>;ver=1.1,</3/0>;ver=1.1"] received on the resource /rd from Identity /<IP_ADDRESS>:60815[psk=leshan100]
org.eclipse.leshan.core.link.LinkParseException: Unable to parse link
        at org.eclipse.leshan.core.link.lwm2m.DefaultLwM2mLinkParser.parseCoreLinkFormat(DefaultLwM2mLinkParser.java:155)
        at org.eclipse.leshan.server.californium.registration.RegisterResource.handleRegister(RegisterResource.java:146)
        at org.eclipse.leshan.server.californium.registration.RegisterResource.handlePOST(RegisterResource.java:104)
        at org.eclipse.californium.core.CoapResource.handleRequest(CoapResource.java:246)
        at org.eclipse.leshan.core.californium.LwM2mCoapResource.handleRequest(LwM2mCoapResource.java:51)
        at org.eclipse.californium.core.server.ServerMessageDeliverer.deliverRequest(ServerMessageDeliverer.java:106)
        at org.eclipse.californium.core.network.stack.BaseCoapStack$StackTopAdapter.receiveRequest(BaseCoapStack.java:218)
        at org.eclipse.californium.core.network.stack.AbstractLayer.receiveRequest(AbstractLayer.java:84)
        at org.eclipse.californium.core.network.stack.AbstractLayer.receiveRequest(AbstractLayer.java:84)
        at org.eclipse.californium.core.network.stack.BlockwiseLayer.receiveRequest(BlockwiseLayer.java:562)
        at org.eclipse.californium.core.network.stack.ReliabilityLayer.receiveRequest(ReliabilityLayer.java:296)
        at org.eclipse.californium.core.network.stack.AbstractLayer.receiveRequest(AbstractLayer.java:84)
        at org.eclipse.californium.core.network.stack.BaseCoapStack.receiveRequest(BaseCoapStack.java:140)
        at org.eclipse.californium.core.network.CoapEndpoint$1.receiveRequest(CoapEndpoint.java:302)
        at org.eclipse.californium.core.network.UdpMatcher$2.run(UdpMatcher.java:289)
        at org.eclipse.californium.elements.util.SerialExecutor$1.run(SerialExecutor.java:290)
        at org.eclipse.californium.core.network.CoapEndpoint$6.run(CoapEndpoint.java:1341)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.util.UnknownFormatConversionException: Conversion = '%'
        at java.base/java.util.Formatter.checkText(Formatter.java:2732)
        at java.base/java.util.Formatter.parse(Formatter.java:2718)
        at java.base/java.util.Formatter.format(Formatter.java:2655)
        at java.base/java.util.Formatter.format(Formatter.java:2609)
        at java.base/java.lang.String.format(String.java:2897)
        at org.eclipse.leshan.core.link.lwm2m.attributes.LwM2mAttributeModel.getApplicabilityError(LwM2mAttributeModel.java:98)
        at org.eclipse.leshan.core.link.lwm2m.attributes.MixedLwM2mAttributeSet.validate(MixedLwM2mAttributeSet.java:97)
        at org.eclipse.leshan.core.link.lwm2m.DefaultLwM2mLinkParser.parseCoreLinkFormat(DefaultLwM2mLinkParser.java:149)
        ... 19 common frames omitted

It seems that the "Object Version" attribute is attached to an object instance (e.g. /1/0) when it is only allowed for the object.
We know that this behavior is not fully consistent with the LwM2M standard but maybe we should allow such devices to register in the Leshan server ?

It seems that the "Object Version" attribute is attached to an object instance (e.g. /1/0) when it is only allowed for the object.
We know that this behavior is not fully consistent with the LwM2M standard

Yep the LWM2M spec v1.1 and v1.0 are clear about that :
See :

http://www.openmobilealliance.org/release/LightweightM2M/V1_1_1-20190617-A/HTML-Version/OMA-TS-LightweightM2M_Core-V1_1_1-20190617-A.html#5-1-2-0-512-Attributes-Classification
http://openmobilealliance.org/release/LightweightM2M/V1_0_2-20180209-A/OMA-TS-LightweightM2M-V1_0_2-20180209-A.pdf

we should allow such devices to register in the Leshan server ?

This is a general question about how Leshan should handle device which doesn't strictly respect the specification.
As I said at https://github.com/eclipse/leshan/issues/1270#issuecomment-1161799929
_"Why be strict by default, In a general way I try to explain this at (https://github.com/eclipse/leshan/issues/1103#issuecomment-937784837 (see answer to ""how should we do that ?"")
So the short idea, trying to be strict by default, but let user make their own server less strict if needed.
I should really take time to create wiki page about this.
In your case you can create your own Parser :
// define your supported attributes
Collection<AttributeModel<?>> supportedAttributes = new ArrayList<AttributeModel<?>>();
supportedAttributes.addAll(Attributes.ALL);
supportedAttributes.addAll(LwM2mAttributes.ALL);

// change the object_version one
supportedAttributes.remove(LwM2mAttributes.OBJECT_VERSION);
// create your custom model which support to assign OBJECT_VERSION To OBJECT and OBJECT_INSTANCE level instance.
supportedAttributes.add(new YourObjectVersionAttributeModel());

// add the new LinkParser
builder.setLinkParser(new DefaultLwM2mLinkParser(supportedAttributes));

Caused by: java.util.UnknownFormatConversionException: Conversion = '%'

This is a bug you should get a message like :

This is fixed in `master`.

I saw a similar problem:
https://github.com/eclipse/leshan/issues/1315
However, after carefully looking at the specification I closed my ticket.
It seems this is an area that will be made more clear in upcoming specifications.
For now, however, it would be helpful if leshan allowed the more compact format ...
There is discussion in the OMA DSME group that would codify the following behavior:
</44>;ver=2.2, </44/0>,</44/1> is OK.
</44>;ver=2.2 is OK (when there are instances of the Object 44).
</44/0>;ver=2.2 is OK.
But </44/0>;ver=2.2,</44/1>;ver=1.9 is not allowed.

I answer to that at : https://github.com/eclipse/leshan/issues/1315#issuecomment-1262387950

Yes it's a good idea. Thanks for explanation.


python312Packages.cgal: 5.6.1.post202403291426 -> 6.0.post202410011635

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.cgal is: CGAL bindings using SWIG
meta.homepage for python312Packages.cgal is: https://github.com/CGAL/cgal-swig-bindings
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 6.0.post202410011635 in filename of file in /nix/store/n2jldmkp1pzikd0nykfxr3mimf30q3vp-python3.12-cgal-6.0.post202410011635

Rebuild report (if merged into master) (click to expand)

9 total rebuild path(s)

9 package rebuild(s)

First fifty rebuilds by attrpath

python311Packages.cgal
python311Packages.graph-tool
python311Packages.gudhi
python311Packages.ifcopenshell
python312Packages.cgal
python312Packages.graph-tool
python312Packages.gudhi
python312Packages.ifcopenshell

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/n2jldmkp1pzikd0nykfxr3mimf30q3vp-python3.12-cgal-6.0.post202410011635 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.cgal https://github.com/r-ryantm/nixpkgs/archive/cdc8907a4add32846e543136d15e7ee6cc70a778.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/cdc8907a4add32846e543136d15e7ee6cc70a778#python312Packages.cgal

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/n2jldmkp1pzikd0nykfxr3mimf30q3vp-python3.12-cgal-6.0.post202410011635
ls -la /nix/store/n2jldmkp1pzikd0nykfxr3mimf30q3vp-python3.12-cgal-6.0.post202410011635/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

4 packages failed to build:

python311Packages.graph-tool
python311Packages.ifcopenshell
python312Packages.graph-tool
python312Packages.ifcopenshell

8 packages built:

python311Packages.cgal
python311Packages.cgal.dist
python311Packages.gudhi
python311Packages.gudhi.dist
python312Packages.cgal
python312Packages.cgal.dist
python312Packages.gudhi
python312Packages.gudhi.dist

Maintainer pings
cc @pbsds for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

this should be a leaf package, fix in https://github.com/NixOS/nixpkgs/pull/347768


Added Insider Trading trigger to Aletheia connector

When submitting a connector, please check the following conditions for your PR.

[X ] I attest that the connector works and I verified by deploying and testing all the operations.
[X ] I validated the swagger file, apiDefinition.swagger.json, by running paconn validate command.
[X ] If this is a certified connector, I confirm that apiProperties.json has a valid brand color and doesn't use an invalid brand color, #007ee5 or #ffffff. If this is an independent publisher connector, I confirm that I am not submitting a logo.

@TimHanewich, please upload the documents to ISV studio.


add iamc filter for tabulate/ list runs

Added iamc filter similar to mp.backend.models.tabulate and mp.backend.scenarios.tabulate to mp.runs.tabulate (all applies to .list methods as well).
Why was this added?
To be able to filter runs by whether they have IAMC data associated with certain properties, e.g. get all runs that have IAMC data for the Cumulative Emissions|CO2 variable or the World region.
usage:
mp.runs.tabulate(
            iamc={
                "variable": {"name__like": "Cumulative Emissions|CO2*"},
                "unit": {"name": "World"},
            }
        )

@danielhuppmann please take a look on whether this is sufficient for the needs for filtering runs on IAMC data that you perceived (for ixmp4-ts it is sufficient).


Issues with running the script

When I try running unlearn_harm.py I get the following error:
pyarrow.lib.ArrowInvalid: Column 5 named input_ids expected length 1000 but got length 1096
after replacing load_dataset's split with "train" because the original split did not exist.

I met the same problem: There is some issue with the map, if the map input rows/batch is 1000 and output is 1096 rows, then the problem popup, one walkaround is write the map manully.
dataloader = DataLoader(dataset, batch_size=1000)
d = {}
d["input_ids"] = []
d["attention_mask"] = []
d["start_locs"] = []
for batch in tqdm(dataloader):
    p_batch = preproccess(batch)
    d["input_ids"].extend(p_batch["input_ids"])
    d["attention_mask"].extend(p_batch["attention_mask"])
    d["start_locs"].extend(p_batch["start_locs"])
dataset = Dataset.from_dict(d)


fix: contract typo

Checklist:

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)

I suppose it is best this does not bump the version. Otherwise, after updating, Campers will likely rebuild the container and lose progress 🤔

That's definitely a disadvantage, justifying keeping the same the version for as long as possible. Maybe there could be some way to semi-or-fully automatically update courses from within container, if only patch version is bumped?


Add debugging challenges to curriculum

An example of what I mean could be something like:
"The below code is supposed to change the monsters health when the user press is the attack button, but there's an issue. Fix the issue so that the attack button works and all tests pass". The issue could be some sort of common error that users make such as typing in the referenced variable incorrectly or writing the functions syntax wrong, etc
Is your feature request related to a problem? Please describe.
Right now we only have one type of challenge in our courses which is  where are you right new code to implement a task. If we add the debugging challenges we get the following benefits:

Breaking up the current monotony of having only one having one type of challenge
Improve peoples' skills with debugging their own code on normal challenges
As curriculum curators we now have an additional tool that we can use to emphasize certain concepts.
it can be a good way to help sharpen people's skills with syntax or help them review a concept

Describe the solution you'd like
Under the hood this can be implemented the same way as normal coding challenges except that the starting code has a wrong version of the solution as opposed to no solution
Describe alternatives you've considered
See "additional context" section below
Additional context
This was discussed on discord. See the messages after https://discord.com/channels/692816967895220344/715074489422970962/1060991198556340364
In the future I think there should be even more types of challenges such as multiple choice challenges (to be clear I'm not talking about heavily relying on this type of challenge, I am talking about using multiple choice challenges occasionally for emphasizing certain concepts). Another type of challenge could be a more freeform, self-evaluation type of code challenge.

This would be blocked until #49018 is resolved - currently we cannot run tests on code with certain errors.

Hmm, I've just left a comment in the issue saying that it shouldn't be an issue/should be easy to resolve if we are only talking about pure JavaScript. Perhaps we can start off with challenges that only involve changing the JavaScript?

Hi oriooctopus.

In the future I think there should be even more types of challenges such as multiple choice challenges (to be clear I'm not talking about heavily relying on this type of challenge, I am talking about using multiple choice challenges occasionally for emphasizing certain concepts). Another type of challenge could be a more freeform, self-evaluation type of code challenge.

Something similar to  "The Little Schemer" (Daniel P. Friedman && Matthias Felleisen)   or more like "Lists and Lists" (Andrew Plotkin)?

@diegoperezm can you clarify?

Sure,  @oriooctopus :+1:  .

"The Little Schemer"(from https://www.crockford.com/little.html, Douglas Crockford):

After reading the book, I was changed. In a good way. There are very few books that deeply change the way that you think. This is one of those books.

The format is a programmed text with questions on the left side and answers on the right. The way you use it is to read a question, think about the question, come up with an answer, and then compare your answer to Friedman's answer.

As you can see this is different from the traditional approach: explain a concept, show an example and expect that the student have the necessary skills to solve the exercise.
The Little Schemer uses a chain of questions-answers to arrive at the code. The idea is to think, come up with an answer, and then compare and  not  memorize facts.

"Lists and Lists" (Andrew Plotkin):
Is a game-tutorial (Interactive Fiction). You try things to solve the exercises, the fact that you have limited options makes it a "guided" tutorial.

Anyway. Another option to add debugging challenges is to take the same approach  as the one used in the forum: ask questions (a sort of checklist).
If we use the forum as a reference, many times asking for help is limited to write "it doesnt'work/"why doesn't it work?". When someone says "why doesn't it work?" it indicates that they don't have the skills to start analyzing the problem. Knowing what to ask is important when learning  to debug.
This can be easily implemented: add to the "Get Help" button a section:  "checklist" or something similar or it can be done in a more complex way: asking them to write function signatures, comments,  unit tests, use a debugger, etc (this is the approach of the book "How to design programs", Matthias Felleisen, Matthew Flatt, Shriram Krishnamurthi).
Have a nice day.

Thanks. Right now I think the courses are setup to have lots of small steps that the user fully solves. I agree that getting the user to think independently on larger problems without the stress of having to get everything right (like what you're saying about seeing the answer after they guess) is useful because it more resembles the real world.

This can be easily implemented: add to the "Get Help" button a section: "checklist" or something similar or it can be done in a more complex way: asking them to write function signatures, comments, unit tests, use a debugger, etc

I think this is already mostly done with hints, no?

I think this is already mostly done with hints, no?

Not really, an example:

freeCodeCamp Challenge Guide: Where do I Belong

https://forum.freecodecamp.org/t/freecodecamp-challenge-guide-where-do-i-belong/16094
Hints:
Hint 1:
The first thing to do is sort the array from lower to bigger, just to make the code easier. This is where sort comes in, it needs a callback function so you have to create it.
Hint 2
Once the array is sorted, then just check for the first number that is bigger and return the index.
Hint 3
If there is no index for that number then you will have to deal with that case too.
Forum
OP:

Hi there. I am a bit stuck. Would like an input, where is the error. Thanks

RandellDawson (Question):

If I have the following:
const arr = [3, 10, 5];
arr.sort()
What does arr look like after the above code executes?

OP:

According to my limited knowledge should be [3, 5, 10].

RandellDawson:

Test the code and see.

As you can see is a different approach.
Also, users do not write unit tests,  nor do they write comments, very few have used a debugger before asking on the forum. In general they only focus on finding the answer to pass the challenge.

We have plans to do this in our upcoming curriculum.


Support HTTP proxies (enhancement)

Some IT infrastructure does not allow to connect to the internet directly but provide HTTP proxies.
Can you either modify the JumblrClient.java class to accept proxy settings or based on this Oracle documentation support the related environment variables, please?

@sawo You can solve it in the following ways:
System.setProperty("http.proxyHost", "<IP_ADDRESS>");
System.setProperty("http.proxyPort", "1080");
System.setProperty("https.proxyHost", "<IP_ADDRESS>");
System.setProperty("https.proxyPort", "1080");


fix: Prevent unexpected device change caused by torch.split function

In parallel GPU mode, the torch.split function can sometimes produce a result tensor on a device different from the input tensor. This can lead to errors such as:
Error generating stream: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:3! (when checking argument for argument tensors in method wrapper_CUDA_cat)

To resolve this issue, this PR proposes assigning the same device as the input tensor to prevent such occurrences.

Thank you


Is there any particular reason the Populate method takes a Registry and not an IRegistry?

Currenty signature is
public static void Populate(this Registry registry, IEnumerable<ServiceDescriptor> descriptors)

Shouldn't it be on IRegistry instead
public static void Populate(this IRegistry registry, IEnumerable<ServiceDescriptor> descriptors)

Just wondering :-)

Yes, there are reasons; the IRegistry interface does not allow you to set policies etc. Just try changing the signature and see what breaks :wink:

I did, it builds fine and the test is still passing :-)
It also seems that public static void Populate(this ConfigurationExpression config, IEnumerable<ServiceDescriptor> descriptors) is not useful either, since ConfigurationExpression inherits from Registry (and hence indirectly implements IRegistry)
Or am I missing something obvious?
We stumbled upon the issue while using that: https://github.com/davidfowl/ServiceProviderFactorySample/blob/master/src/ServiceProviderFactorySample/Extensions/StructureMapExtensions.cs, which was adapter to use IRegistry/Registry. But this is all useless now because of this project ;-)

The reason the ConfigurationExpression overload exists is because that was the initial method. The Registry overload was introduced later, but we didn't want to break people. I guess we could move everything to IRegistry and do a major version bump.

Yes, I was going to say that this would be a (binary) breaking change.
I'll do a small PR that you can include when you'll feel like making a new major version :-)
Thanks!


Docker registry REST API V2 not writing GET blob content to local filesystem correctly from java program

I'm trying to use Docker Registry HTTP API v2 as per this document https://docs.docker.com/registry/spec/api/#pulling-an-image to pull series of blobs for a given image in Java RESTTemplate service written in maven. However, when i use GET /v2//blobs/ API to retrieve particular blob layer for example, hello-world image with sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 blob
GET http://localhost:5000/v2/hello-world/blobs/sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
which returns size 32 bytes (with 42Characters) using java RestTemplate. While writing to a local file system, its writing as 42 bytes instead (with 42 characters).
Looking deeper looks like docker registry is storing each blob layers internally in Gzip archive (application/gzip) format however "Content-Encoding" header is not specified in the docker REST response for the above mentioned GET API. The problem is that the local file system that java program writes to is storing as "application/octet-stream" format thus showing as 42 bytes size.
Does anyone have any pointers/suggestion to workaround this issue? Perhaps i may be missing something. Any inputs much appreciated.
Directory of docker registry container where images folder is mounted to the local file system. For example, output for sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 as file type "Gzip archive (application/gzip)"
-rw-r--r--. 1 root root 32 Oct  3 18:15 data
[a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4]$ file data
data: gzip compressed data```

Directory where java program writes output files for example, sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 as file type "Binary (application/octet-stream)"

```[ sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4]$ ll
total 4
-rw-rw-r--. 1 doss doss 42 Oct 17 13:16 data
[sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4]$ file data
data: data```

@swepnadoss It looks like whatever web framework you're using is trying to be crafty about Content-Encoding. For serving blobs, we leave compression up to the content. I'd recommend disabling any content-type handling or automatic compression in the web framework. Without seeing the code, I am not sure if we can provide more help.
I'd recommend reaching out to the mailing list for the web framework you're using to get more help. This isn't really something we can provide help with except to provide direction on the intended response.


TikTok request: download using a user profile URL

Checklist

[x] I'm reporting a site feature request
[x] I've verified that I'm running youtube-dl version 2019.12.25
[x] I've searched the bugtracker for similar site feature requests including closed ones

Description

When using youtube-dl for YouTube, you can use the URL of a user's profile and it will automatically download ALL videos uploaded by that user.
It would be nice if we had that same functionality for TikTok. An example of a profile URL for TikTok would be [https://www.tiktok.com/@buzzfeed].

Tiktok user page returns latest 24 videos only. Making api calls to tiktok requires generating _signature field, which may be difficult to implement and unable to maintain (due to frequent changes).


Live demo only updates the head position

Just so You know
the head is spazzing a bit, but thats probably because of poor lighting.
Hands and legs are not moving at all.

Noticing the same issue here.

check the experimental branch


Can you add autocomplete function?

such as this, get the prompt data from backend.

Hi! I still have to finish some stuff and be sure that the componen is stable. Once I finish with that I would like to implement this. Thanks for the suggestion.

THX, I am using your component in my project.

Can I suggest to leave this issue open to keep track of the work for autocomplete?

further to autocomplete it would be nice if validators can talk to a backend. e.g imagine tags being products codes and a validator being able to talk to the backend using an axios/ajax call and return trey or false if the product exists or not.
nice work on this component.

Hi. Thanks for your work on this component.
I also wanted an autocomplete functionality but i would like to compose a custom component with this one to achieve the same. However, i need a callback on the event of input change, so that i can use the partial input text to generate a list of autocomplete options. Can i send a pull-request for this change? Should be very simple.

sure :)

Autocomplete / callback on-change functionality would be super cool 👍

Basically just need a @keyup functionality for this right? With that as a minimum, we could implement autocomplete in our client relatively straightforward.

I might make a fork of this to work on this since I require it for a project but don't wish to reinvent the wheel.
Also since its just used for tags I don't think an http call is required since most of the time you are just checking if tags exist. maybe just pass it an array like the pre filled ones then checking if the value exists in array if so suggest it, to start with at least.

@michaelmano  think of tags being products codes and products being tens of thousands, does not make sense to give it an array of thousands of valid product codes but make much sense to make an Ajax call to validate if the code exists! if the validation is implemented in some sort of function callback then its up to you to simply check an array or make an Ajax call depending on your needs!

@unitedworx then maybe have either option. An auto complete array for smaller things and an auto complete URL that will post with a denounce function to get an array

Autocomplete is must have feature, please consider to add it.

Mmmmh this component is really useful.  Too bad it's not 100% finished, it lacks a few methods you don't seem to be willing to implement I guess I'll have to do it on my own ^^'

@LorrainInfinity Sometimes is really hard to make enough time to maintain an open source project but PR's are welcome 😉

@ianaya89 Yes that's true ^^. It doesn't lack much and I think it could be useful enough to a lot of people but I'm trying to use it for a very complex architecture so I'll have to improve it a bit :p

@LorrainInfinity besides this feature, what else is missing in your opinion?

It'd be great to be able to detect which element was deleted.  I know you did add a update:tags method (btw you should add it on the home page of the git project so people know about it :p).  But I need to know exactly what element is removed from the array of elements.
On another hand it'd be great to be able to pass an array of objects as tags, and if so, add a "track-by" prop so it tracks based on a specific key.  That's because I need to create an array of strings instead of using my real array of objects.

@LorrainInfinity You might check out Vuetify which pretty much has all these features in their multi-select. https://vuetifyjs.com/en/components/selects#api

Mmh yes maybe it could work but what I really need here is a tag creator not a multiselect (even though I saw that there is a "tag" option to allow tag creation, but in the end, it'll still be a select list isn't it?).
Anyway thank you for the component I'll look into that :)

You can use it as a multi-select but also add new tags.

Yea this feature will be very useful.
I am happy to contribute my solution after I've tried hacking it

I have added this PR to allow implement autocomplete out of the component:
https://github.com/matiastucci/vue-input-tag/pull/63


Scientific memes hackathon

An event to create a collection of memes that explain common reasoning errors, and little write ups about each one, how to use it, its scope etc. We give each other feedback and find the best ones and use them & spread them.

https://x.com/mold_time/status/1856014149110603860

“mock dataset” strikes me as unusually hazardous

https://x.com/twiz19071051/status/1857262277180690440

I want this but in a format I can edit to put other things in there. instead of "use government to control" you can rewrite that to be "those who cheat others, those who put their own gain over others, those who don't create value but extract it from others who work hard"
https://x.com/DefenderOfBasic/status/1852547755853419002

https://x.com/DefenderOfBasic/status/1859430644847124949


:technologist: set that nft can be burned

TL;DR
I realized that UI does not show anyhow that the nft is burned
And I can do whatever (it fails on-chain)

Screenshot 📸

[x] My fix has changed something on UI; a screenshot is best to understand changes for others.

Copilot Summary

🤖 Generated by Copilot at b12a9a4
This pull request enhances the GalleryItem component to display burned NFTs differently. It uses a fire emoji and hides some elements to indicate that the NFT is no longer available.
copilot:poem

@exezbcz let's make this PR dissapear?

ref #4612


Create test case infrastructure

Create test cases that test:
inheritance/templating
overrides
application usage
Create a script to automate test cases, and verify results.
Check containers created, what ifaces each have, what links they have (via ping?)

sufficient test cases are in place for v0.1, need to automate for future release

Automation focus for v0.2


Add a

Adding a  element to the HTML wrapper, for ease of understanding what's in what tab in a browser

Thanks James.


Adjust filter operands API

Turn { id_gt: "my-id" } into { id: { gt: "my-id" } }

@timsuchanek is this done?

Yes!


Savemenu - Delete Save Data often doesn't delete the save_data.json file

I've been using the SaveMenu item in my project, and many time the button : Delete Save Data fails to work.
It seem to work a few times, then completely stops working ( perhaps after restarting Unity?
I now always have to delete the file manually.

Unity version [e.g. 2019.2.10f1

Additional context
I'm available to help resolve this issue.

Having a look over the existing save code, the only thing that tries to delete the file from storage is the Restart function from the SaveMenu, it also clears all history and notifies of a save reset.
A quick test of the save demo scene doesn't reproduce the issue.
As noted here the save system is probably being replaced, but any additional information is appreciated.

There is a Delete Save Data button in the inspector when selecting the SaveMenu object.
while it did work a few times, it stops working after a while.. it's pretty easy to reproduce on my side.
Load the project.
Make sure there is a save file in the User folder.
Press Delete Saved Data.
Result : The file is not deleted.
It's not a big issue, as I can delete the file manually.


Improve Getting Started experience

[ ] Setup npm create svelte-ux@latest

Templates

Basic / Barebones
Starter (with AppLayout / navigation)

[ ] Support opening in

[ ] StackBlitz

https://developer.stackblitz.com/guides/integration/open-from-github
https://developer.stackblitz.com/guides/integration/create-with-sdk

[ ] SvelteLab
[ ] CodeSandbox
[ ] Standard REPL (if possible)

[ ] Update documentation

See also: @jycouet's REPL


🔧 Fix Ruff configuration unintentionally enabling and re-disabling mccabe complexity check

Enabling "C" turns on complexity checks (C90, mccabe), which is unintended. Instead, enable "C4" to get flake8-comprehensions checks
See docs at https://docs.astral.sh/ruff/rules/#flake8-comprehensions-c4

Nice, thanks @jiridanek! 🚀