Fail to reindex an index with slave

Getting exception when calling reindex():
[AlgoliaSearch\AlgoliaException]
A slave index is already slave of another index
It appear to be calling setSettings() before unsetting the "slaves" key

        if (count(array_keys($settings)) > 0) {
            $index->setSettings($settings);
        }

        if ($b && isset($settings['slaves'])) {
            $b = false;
            unset($settings['slaves']);
        }

Please review this issue.

Yes this is intended because we want to set the slaves for the first index of getIndices. I think it can happen when modifying the slaves name between classes. The easiest way to go is to delete slaves indices from the algolia dashboard

Can you tell me if that fixes your issue


Add a variable to control Tesseract's dpi parameter

Tesseract has the parameter --dpi for images which lack DPI information in their meta data. To avoid that Tesseract hast to guess the image's DPI and to suppress the subsequent warning, it might be useful to add a corresponding variable to the Makefile.

You can (mis-)use the existing PSM parameter. :-)
Example: PSM=6 --dpi 300
I think that the image resolution is not used here, so Tesseract issues an unnecessary warning that should be removed.

So, I can safely ignore the warning (and the often incorrect estimation of an image's DPI)?

Yes for training. No for recognition. Hm, while writing this answer I just think again about training: it runs recognition on the evaluation set. Up to now I only examined the initial training phase. The evaluation phase still needs a closer look whether it uses the DPI information.

I forgot something. Training and evaluation both use lstmf files (which include the image and the text). According to my tests, those lstm files don't change when I pass different --dpi arguments. That implies that the complete training process does not use the DPI, so you can safely ignore the warning.

@stweil Do the lstmf files change when different --psm values are used?

@Shreeshrii Most likely not:
(env) $ tesseract DEWARP_0006_l782.tif DEWARP_0006_l782 --psm 13 --dpi 300 lstm.train
Tesseract Open Source OCR Engine v4.1.0-rc1-223-g3e71 with Leptonica
Page 1
(env) $ ls -l DEWARP_0006_l782.lstmf
-rw-r--r-- 1 kmw kmw 173 Oct  9 12:50 DEWARP_0006_l782.lstmf
(env) $ tesseract DEWARP_0006_l782.tif DEWARP_0006_l782 --psm 7 --dpi 300 lstm.train
Tesseract Open Source OCR Engine v4.1.0-rc1-223-g3e71 with Leptonica
Page 1
(env) $ ls -l DEWARP_0006_l782.lstmf
-rw-r--r-- 1 kmw kmw 173 Oct  9 12:50 DEWARP_0006_l782.lstmf


Add info internship 2024 01

@EverVino could you review this PR please?

LGTM! We have an clear schedule for internships. Thank you @xmnlab
Now the makim pages.preview command is working as well.


Use a pool of timers when waiting for query

Required for all non-trivial PRs

[x] Rebased/mergable
[x] Tests pass
[ ] CHANGELOG.md updated
[x] Sign CLA (if not already signed)

waitForQuery is called with every query the executor receives. In the case of setups where there is a non-zero QueryTimeout, each call to waitForQuery currently initialises a new time.Timer.
This has an overhead of 3 allocations per call.
This commit adds a sync.Pool to the QueryExecutor, which provides a
thread-safe free list of time.Timers, which will grow and shrink in size
as necessary.
@jsternberg whether you want to accept this PR is probably dependent on how important a saving of 3 allocations per query is, if the user is running Influx with QueryTimeout > 0. 😄

I think I'm going to close this for now and reopen if the query timer ever becomes a problem. Initial testing of this PR without configuring influx_stress any resulted in a slower average query performance:
jsternberg:[~/g/s/g/i/influxdb] $ (master $=) influx_stress
Total Requests: 2000
        Success: 2000
        Fail: 0
Average Response Time: 99.967317ms
Points Per Second: 357273

Total Queries: 250
Average Query Response Time: 41.037655ms

jsternberg:[~/g/s/g/i/influxdb] $ (master $=) influx_stress
Total Requests: 2000
        Success: 2000
        Fail: 0
Average Response Time: 104.864841ms
Points Per Second: 343667

Total Queries: 250
Average Query Response Time: 43.454234ms

The first is master, the second is this PR. Now the difference between these isn't a lot and it would require further testing to determine if this slowdown was from randomness or an actual slowdown. My initial guess is likely that the allocations are cheaper than the locks needed for the pool of timers. It might make a difference if this is amortized over time, but further testing would be needed to confirm that and I'm not sure it's worth it.

@jsternberg I expected it to be a bit slower for very low usage such as 250 queries. And of course it will behave identically to current master if the default query timeout of 0 is in play.


feat(collapsiblePanel): added additional properties

Simple Design property added
Header Background Color property added
Body Background Color property added
Radius property added
Link

PS: @Lukeybooi, please avoid usage of styled-components if the same functionality can be implemented using antd-style


riverpod code updated

@govindmaheshwari2 Add reference of HMSSDK released on pub.dev in pubspec.yaml file


CmsBlock and CmsElement computed, @input and @change not working

PHP Version
8.3
Shopware Version
<IP_ADDRESS>
Expected behaviour
The text field should render the value from the config. When the value gets updated by the user the value should get updated in the config.

The computed setter should be triggered on each input change.
The @input event should be triggered on each input change.
The @change event should be triggered on value change.

Your example from the docs:
index.js
computed: {
        dailyUrl: {
            get() {
                return this.element.config.dailyUrl.value;
            },

            set(value) {
                this.element.config.dailyUrl.value = value;
            }
        }
    },

methods: {
    createdComponent() {
        this.initElementConfig('dailymotion');
    },

    onElementUpdate(value) {
        this.element.config.dailyUrl.value = value;

        this.$emit('element-update', this.element);
    }
}

html.twig
{% block sw_cms_element_dailymotion_config %}
    <sw-text-field
          v-model="dailyUrl"
          class="swag-dailymotion-field"
          label="Dailymotion video link"
          placeholder="Enter dailymotion link..."
          @input="onElementUpdate"
          @change="onElementUpdate">
    </sw-text-field>
{% endblock %}

Actual behaviour

The default value from the config is displayed correctly
The computed setter is never triggered on user input
The @input event is never triggered on user input
The @change event is never triggered in user input
On @blur the value is reset to the default value

How to reproduce
Use the examples from your own docs:
https://developer.shopware.com/docs/guides/plugins/plugins/content/cms/add-cms-block.html
https://developer.shopware.com/docs/guides/plugins/plugins/content/cms/add-cms-element.html
For text input my current workaround is using the @blur event to trigger onElementUpdate . But this is not working for select or radio elements. This makes those elements unusable at the moment.

Seems like the docs are outdated, this is a Vue2 to Vue3 migration thing. With 6.6 it should be used something like this:
    <sw-text-field
          v-model:value="dailyUrl"
          class="swag-dailymotion-field"
          label="Dailymotion video link"
          placeholder="Enter dailymotion link..."
          @update:value="onElementUpdate">
    </sw-text-field>

thanks for bringing this up, I made an update for the docs :v:
https://github.com/shopware/docs/pull/1359


Triggering script from Tasker while screen is off

Hey,
Using version 1.0.3 from the Play Store I've setup a script to run via Tasker which will almost always run while the screen is off. The task runs and launches the console, but the script doesn't run until I open the console. The script runs and exits successfully, but it would be great to have the script run when the screen is off.
Any thoughts or limitations imposed by Android on this?
Thanks!

Hey,
Using version 1.0.3 from the Play Store I've setup a script to run via Tasker which will almost always run while the screen is off. The task runs and launches the console, but the script doesn't run until I open the console. The script runs and exits successfully, but it would be great to have the script run when the screen is off.
Any thoughts or limitations imposed by Android on this?
Thanks!

I was trying to do the same thing and I found that you can actually do it with the "Take WakeLock" option that appears touching the 3 dots you can see when de console is running. You choose that and you will se that it now says "Drop Wakelock" leave like that and then turn the screen off. It's important to to not swipe out the app from background processes, this way it doesn't work.
Hope this is usefull more than a year later, sorry for bad english, second language. Have a nice one!!


downloads require running autoreconf -i and autoconf undocumented

I may have missed something but if one downloads either with git or the "release" tarballs configure is not available so one must run
autoreconf -i
autoconf
before running configure. This seems does not seem to be documented anywhere.
Couldn't you provide tarballs with the configure already there?

Putting configure "already in there" somewhat defeats the purpose of autoconf, as I understand it. Using it should be as simple as running:
autoconf
./configure

from the root folder of the project. That said, I just tried master on a recent Ubuntu and I was getting some warnings so it doesn't appear to be working properly at the moment. This is something I need to start checking with more diligence on release tags.
@qdii @magicant @LocutusOfBorg @bittwiddler1 any of you have a moment to look at this? I won't get to it for a few days, and you have all contributed at one time or another to the autoconf support.

I'm ok with having to do something before the configure if neccessarily but the README starts with
./configure
and makes no mention of running something before which seems problematic.
Thanks

I'm unclear on the particulars of your previous comment. ./configure is not mentioned in the README; only CMake (the primary supported build system) is mentioned there.

Hmm, I downloaded unittest-cpp-1.5.0  and the INSTALL file states:
Briefly, the shell commands `./configure; make; make install' should
configure, build, and install this package.
Am I accidently in the wrong universe?
Thanks for answering so promptly?
Should switch to 1.51?

Ah, I was looking at the README. Now that I see what you're saying, I'd say the INSTALL file is out of date.
If you must autotools, then I'm afraid things appear to be broken at the moment; I'd be happy to see a PR from you or anyone else on this as I doubt I'll have the cycles for a few days.
If you can use CMake, then you can should be able to install using the instruction me here: https://github.com/unittest-cpp/unittest-cpp/wiki/Building-Using-CMake

Ahh, running autoreconf on my Apple worked fine but it looks like my autoconf on my linux system is out of date hence could not generate ./configure.
I'll try the cmake.

I don't particularly love CMake either, but I am not interested in reopening that conversation at this time.
As previously stated, I'll happily accept PRs on the autotools support files -- so I wouldn't say the project has abandoned it. Sorry for the inconvenience.

I'm trying the commit 5b34768df0c4552aeac1ef980b683b15a3b473f2 on Fedora 23 with Autoconf 2.69, and don't find any particular issue about autoconf. The commands
autoreconf -i
./configure
make

just work for me.
@pjohnmeyer Are you running autoconf directly? I recommend using autoreconf since I automatically runs other autoxxx commands if necessary.

@magicant I was, switched to your method as above [Ubuntu 15 and autoconf 2.69] and got some errors. Realized that I didn't have libtool installed -- I hadn't really used this VM for dev until now.
@BarrySmith, can you see if it works this way for you? If so, this is just a documentation problem.

Doing this:
autoreconf -i
./configure
make
Works for me, I'm using arch linux.

From what I've experienced with GNU auto tool packages,  they usually ship with a pregenerated configure shell script...
If the makefiles support it, (and if I understand correctly) running make dist should generate a .tar.gz with the configure script at the root.

Im preoccupied with work and family but I might be able to take a look tomorrow

On Feb 9, 2016, at 10:29 AM, Tiberio Menezes<EMAIL_ADDRESS>wrote:
Doing this:
autoreconf -i
./configure
make
Works for me, I'm using arch linux.

This works fine for my on my Apple Mac (latest Xcode) but on linux-Ubuntu_12.04-x86_64  with autoconf-2.69 automake-1.15 libtool-2.4.6/bin   I get
$ autoreconf --force --install -v
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
aclocal: warning: couldn't open directory 'm4': No such file or directory
autoreconf: configure.ac: tracing
autoreconf: configure.ac: not using Libtool
autoreconf: running: /homes/petsc/soft/linux-Ubuntu_12.04-x86_64/autoconf-2.69/bin/autoconf --force
autoreconf: running: /homes/petsc/soft/linux-Ubuntu_12.04-x86_64/autoconf-2.69/bin/autoheader --force
autoreconf: running: automake --add-missing --copy --force-missing
UnitTest++/Makefile.am:1: error: Libtool library used but 'LIBTOOL' is undefined
UnitTest++/Makefile.am:1:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'
UnitTest++/Makefile.am:1:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.
UnitTest++/Makefile.am:1:   If 'LT_INIT' is in 'configure.ac', make sure
UnitTest++/Makefile.am:1:   its definition is in aclocal's search path.
Makefile.am:4:   'UnitTest++/Makefile.am' included from here
autoreconf: automake failed with exit status: 1
But if I run libtoolize first then run autoreconf --force --install it works! I have no idea why.

—
Reply to this email directly or view it on GitHub.

@pjohnmeyer unfortunately I'm a huge CMake fan, and I really don't understand how autotools works, IIRC I just did a fix in a pkgconfig file, but I try to stay away from autotools issues whenever possible. So for me +1 for CMake switch :)

@LocutusOfBorg AFAIK, unittest-cpp is already primarily using CMake for builds, so there's no need!
However, I was under the impression the GNU Autotools files were here for those who don't have CMake installed on their system. I know CMake is an awesome tool, but it's also much more dependency-heavy than the GNU Tools, which mostly depend on text-processing programs.
Here's a high-level overview of how the auto tools are supposed to work:

Running autoconf generates a configure shell script using configure.ac as a template.
Running configure runs dependency checks on your system (much like cmake ..) and then calls automake.
automake generates a Makefile at your project root using Makefile.am as a template.
make builds your code using the Makefile

If you really need good make and Visual Studio support, though, CMake is the way to go.

Putting configure "already in there" somewhat defeats the purpose of autoconf, as I understand it.

@pjohnmeyer It actually depends on who's compiling the package.  For the purposes of avoiding clutter in your source control and the source tree, you don't generally check in the configure script, but from what I've seen in most .tar.gz files, the release distributions of the source do contain a configure script.
From Wiki:

Autotools assists a software development person or team to write cross-platform software and make it available to a much wider user community, including in its source code form to those users who wish to build the software themselves. In most cases users simply run the supplied configure script (which has no dependencies other than the presence of a Bourne-compatible shell), and then a make program.[5] They do not need to have the Autotools themselves installed on the computer.

I may have missed something but if one downloads either with git or the "release" tarballs configure is not available so one must run
autoreconf -i
autoconf
before running configure. This seems does not seem to be documented anywhere.

From where I stand, @BarrySmith's issue is mostly an issue of distribution practices and documentation. :+1:
If it were me, I'd just make sure the release tarball has a ./configure at the root, and/or else document it that people building from github without CMake need have the autotools on their systems in order to generate configure

Thanks for all the input, everybody, I'm the first to admit I'm a little light on autotools knowledge. @BarrySmith : The Libtool library used but 'LIBTOOL' is undefined error you saw was the same thing I saw before I installed libtool on my fresh Ubuntu install.
To this point, I've only been tagging code -- I haven't been creating true releases through GitHub. To that extent, 1.5.* releases have been source distributions only.
If preparing an autotools distribution would make life significantly better for the Linux dev community, I'd be happy to spend a little more time on it. The main effort would be in writing a release script, which I need to do anyway. I haven't decided precisely how I want to solve this "problem" though; so I'm gonna put a few more days of thought into it.

Thanks for all the input, everybody, I'm the first to admit I'm a little light on autotools knowledge. @BarrySmith : The Libtool library used but 'LIBTOOL' is undefined error you saw was the same thing I saw before I installed libtool on my fresh Ubuntu install.
To this point, I've only been tagging code -- I haven't been creating true releases through GitHub. To that extent, 1.5.* releases have been source distributions only.
If preparing an autotools distribution would make life significantly better for the Linux dev community, I'd be happy to spend a little more time on it. The main effort would be in writing a release script, which I need to do anyway. I haven't decided precisely how I want to solve this "problem" though; so I'm gonna put a few more days of thought into it.

So, I've spent some more significant time with the autotools files tonight because on #95 I had previously neglected to updated the Makefile.am files with the new files. As it turned out, I needed not only to update them, but add some m4 macros along with an update to configure.ac to optionally enable C++11 support
I would personally prefer to only support CMake, but I appreciate the use case for autoconf. I'm going to experiment a bit more before addressing this thoroughly.

As you all may have noticed I went ahead and tagged v1.6.0 tonight without having resolved this. However, I intend to address these issues in March and will most likely include, with that, uploading distributions for older tags as well, if it doesn't prove to be too difficult.

When I release 2.0, as with 1.6.1, the ./configure file will be included in the tarball. I am also adding a small note to the top of the install file to indicate that if you're building from source you'll need to run autoreconf. I won't be going back and fixing old download packages.


Update action with configuration options

Updating the action to use configured git environment variables according to vanilla semantic release and official git documentation.

And btw the documentation you are referring to is the node semantic release and not python semantic release. Our documentation is on https://python-semantic-release.readthedocs.io/en/latest/

@relekang does that look right now?
And yeah, I was under the impression that the projects were related in some way but I must have misunderstood.
We're trying to just make sure that we can create a commit and release signed by our service user rather than using the general actions user.

Thanks @cmitzel-ncino 🙌


EmbeddedKafka.stop() fails to shutdown Zookeeper

Using "net.manub" %% "scalatest-embedded-kafka" % "1.1.1" I'm seeing an issue with stopping the embedded servers if the servers have been started individually.  When I start the servers by calling start() the subsequent call to stop() works as expected.  However, when starting each server individually, a call to stop() results in repeated Connection refused ZooKeeper errors .
Looking at the code, the discrepancy can be explained by the fact that start() does not add the EmbeddedZ instance to servers and is thus never stopped by a call to stop() while startZooKeeper() adds the instance to servers.
This is best demonstrated in the REPL (I canceled the session after two errors but they repeat indefinitely):
[info] Starting scala interpreter...
Welcome to Scala 2.12.6 (OpenJDK 64-Bit Server VM, Java 1.8.0_172).
Type in expressions for evaluation. Or try :help.

scala> :paste
// Entering paste mode (ctrl-D to finish)

import net.manub.embeddedkafka.EmbeddedKafka
import scala.reflect.io.Directory
import org.slf4j.LoggerFactory

val log = LoggerFactory.getLogger("REPL-logger")

log.info("EmbeddedKafka.start()")
EmbeddedKafka.start()
Thread.sleep(500)
EmbeddedKafka.stop()
Thread.sleep(500)

log.info("EmbeddedKafka.startZooKeeper()")
EmbeddedKafka.startZooKeeper(Directory.makeTemp("zookeeper-logs"))
Thread.sleep(500)

log.info("EmbeddedKafka.startKafka()")
EmbeddedKafka.startKafka(Directory.makeTemp("kafka-logs"))
Thread.sleep(500)

log.info("Stopping servers")
EmbeddedKafka.stop()
log.info("Servers stopped")

// Exiting paste mode, now interpreting.

INFO  REPL-logger - EmbeddedKafka.start()
ERROR o.a.z.s.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
WARN  k.s.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-logs3439956962492633719.tmp/meta.properties
WARN  k.s.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-logs3439956962492633719.tmp/meta.properties
ERROR o.a.z.s.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
INFO  REPL-logger - EmbeddedKafka.startZooKeeper()
ERROR o.a.z.s.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
INFO  REPL-logger - EmbeddedKafka.startKafka()
WARN  o.a.z.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
  at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
WARN  k.s.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-logs5375698435158631498.tmp/meta.properties
WARN  k.s.BrokerMetadataCheckpoint - No meta.properties file under dir /tmp/kafka-logs5375698435158631498.tmp/meta.properties
INFO  REPL-logger - Stopping servers
ERROR o.a.z.s.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
WARN  o.a.z.ClientCnxn - Session 0x164d250d4e10000 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
  at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
WARN  o.a.z.ClientCnxn - Session 0x164d250d4e10000 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
  at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
  at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
  at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
  at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)

Sorry to comment on a closed issue, but not sure how to get around it.  I've just built and tested master and am still seeing the same issue.  #152 didn't address the underlying issue of the process hanging when stopping ZooKeeper and Kafka after starting each service independently (startZookeeper and startKafka.

I think this was fixed in the new WIP repo...


[SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate signature failure (_ssl.c:1131)')]

I followed the readme.md to install , but it can not work. Errors like below,how should I do?
I already installed openssl.system was ubuntu 20.4
`(venv) pool@plot-machine:~/pool-reference$ sudo ./venv/bin/python -m pool
Traceback (most recent call last):
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 969, in _wrap_create_connection
return await self._loop.create_connection(*args, **kwargs)  # type: ignore  # noqa
File "/usr/lib/python3.8/asyncio/base_events.py", line 1050, in create_connection
transport, protocol = await self._create_connection_transport(
File "/usr/lib/python3.8/asyncio/base_events.py", line 1080, in _create_connection_transport
await waiter
File "/usr/lib/python3.8/asyncio/sslproto.py", line 529, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib/python3.8/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.8/ssl.py", line 944, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate signature failure (_ssl.c:1131)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/pool/pool-reference/pool/main.py", line 2, in
main()
File "/home/pool/pool-reference/pool/pool_server.py", line 291, in main
asyncio.run(start_pool_server())
File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
return future.result()
File "/home/pool/pool-reference/pool/pool_server.py", line 260, in start_pool_server
await server.start()
File "/home/pool/pool-reference/pool/pool_server.py", line 58, in start
await self.pool.start()
File "/home/pool/pool-reference/pool/pool.py", line 188, in start
self.blockchain_state = await self.node_rpc_client.get_blockchain_state()
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/chia/rpc/full_node_rpc_client.py", line 27, in get_blockchain_state
response = await self.fetch("get_blockchain_state", {})
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/chia/rpc/rpc_client.py", line 41, in fetch
async with self.session.post(self.url + path, json=request_json, ssl_context=self.ssl_context) as response:
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/client.py", line 1117, in aenter
self._resp = await self._coro
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/client.py", line 520, in _request
conn = await self._connector.connect(
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 535, in connect
proto = await self._create_connection(req, traces, timeout)
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 892, in _create_connection
_, proto = await self._create_direct_connection(req, traces, timeout)
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 1051, in _create_direct_connection
raise last_exc
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 1020, in _create_direct_connection
transp, proto = await self._wrap_create_connection(
File "/home/pool/pool-reference/venv/lib/python3.8/site-packages/aiohttp/connector.py", line 971, in _wrap_create_connection
raise ClientConnectorCertificateError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorCertificateError: Cannot connect to host localhost:8555 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate signature failure (_ssl.c:1131)')]
^CException ignored in: <module 'threading' from '/usr/lib/python3.8/threading.py'>
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 1388, in _shutdown
lock.acquire()
KeyboardInterrupt: `

Are you running the full node on the same network as the pool?
If you run testnet9 pool, you should run tesntet9 node

Are you running the full node on the same network as the pool?

yes.same machine.

If you run testnet9 pool, you should run tesntet9 node

I want to run it at mainnet

same error


Black formatting

Is your feature request related to a problem? Please describe.
Make code formatting consistent by using black.
Describe the solution you'd like
There are several formatting standards across the sktime codebase. This means we can't use automatic code formatters without making more changes than intended. black is a popular formatter that is used by an increasing number of projects including pandas. It enforces a strict subset of PEP8 standards easily and automatically.
As well as being a simple command-line tool, it can be integrated into several IDEs giving developers lots of flexibility. It can also be made into a git web hook or a unit test that will force contributors to make their code black-compliant to pass the regression tests (here's an example from a side project of mine).

I'm generally in favour of the idea of enforcing consistent coding styles. I'm not sure if we want to do it at this stage because it adds an additional hurdle for contributors, old and new alike.
If we want to enforce it at a later stage, how easy is it to automatically reformat existing Python code to comply with black style? I know that you can just run black <file> but does that generally work without problems?

Yes, it's incredibly easy. You can just run black <file> or black <directory> (to reformat everything at once. If you've integrated it into your IDE, you can reformat the files you have open with a single command/click.
Black is technically beta software but in practice it's extremely stable. (The beta status is only because they may make minor changes to the formatting style in the future). By default it compares the AST before and after formatting to make sure the code isn't effected.
The following notable open-source projects trust Black with enforcing a consistent code style: pytest, tox, Pyramid, Django Channels, Hypothesis, attrs, SQLAlchemy, Poetry, PyPA applications (Warehouse, Pipenv, virtualenv), pandas, Pillow, every Datadog Agent Integration, Home Assistant.

Opted for flake8 for now, changed in #246, but happy to change it to black if someone wants to implement it :)


Cell Refactoring

Cameron and I have some thoughts about how we should refactor Cell a bit.
Cameron thinks that cells should directly work with the Cell Graph Manager in order to get added to cell groups.
Cell Graphs will likely be sim-centric; eg. we might have a CircuitCellGraph, ThermalCellGraph, ShaftCellGraph, etc.

Cell Provider refactoring will also probably be required so that a function is called on the cell and the cell returns the connection mapping with the type of the simulator connections on each side.

Cameron also wants to investigate whether this part of Minecraft is threaded; @bloxgate do you have any idea how this might work?

Last time I really looked into it there were only the game and network threads. That said, there's nothing really forbidding us from making another thread for our own stuff.


Correct FOL-BC-Ask Fig and Page no. readme.md

Corrected FOL-BC-Ask Fig no. from 9.3 to 9.6 and page no. from 332 to 338

@ritwik12 Make sure you are using the International version as a reference. I have a copy of the Indian Subcontinent version and usually, the page numbers are not consistent with the repository.


1.5 165229915

To be merged into 1.5

I think this pull request was accidentally done against the master branch when it was supposed to be done against the 1.5 branch. We're making a new PR.


Implemented flowOn and improved error handling

          1 Warning

:warning:
Branch name should have release/, hotfix/, fix/, housekeep/ or feature/ prefix.

  Generated by :no_entry_sign: Danger


Scheduler scheduled replica to the same node

This issue is due to disk status update delay. One of the node is not available for scheduling, the other two node status didn't update immediately after scheduling replica, it will cause scheduler schedule 2 big replica on the same node, which is impossible in normal situation.

This shouldn't be possible since all replicas scheduling are in an one thread process. And update to the rs inside controller should prevent this.

@yasker Yeah, but scheduler depend on disk status which is updated by node controller. If we schedule replica before node controller update disk status, this situation will occur.

Let node controller watch replicas,

Add a ScheduledReplica map[string]int64 to the Disk.Status.

Change the scheduling logic that each replica should wait for all the other scheduled replicas (`nodeID != "") of the same volume been accounted for before continuing scheduling. Just search for the disk status for the node the previous replica has been scheduled and see if it's name is there.

Fixed.


Change to workshopper

Hello @sethvincent I am asking even though I wonder if its futile: Do you think its hard to change the system to workshopper. The reason why I asked is: It is quite some work to translate the content into various languages. Changing to workshopper would make it easier to translate javascripting to other languages.

:+1:

I'd be +1 on this, i'd love to get rid of some of the hacks i've done.

:+1:

Closing in favor of PR https://github.com/sethvincent/javascripting/pull/105


Fuzzing against the WHATWG spec

Since the goal of this module is to provide a reimplementation of the WHATWG URL Standard (but using different algorithms), it might be worthwhile to use a fuzzer to help discover cases where they are not in alignment. This might help finding more cases such as #6.

I wasn't able to find a good fuzzer for JavaScript programs (there are plenty for JavaScript engines), but there are quite a few good ones for some other languages:

C/C++: afl
C/C++: libFuzzer
Go: go-fuzz / gofuzz
Go: https://golang.org/s/draft-fuzzing-design
Rust: cargo-fuzz
Rust: afl.rs

Fuzz tests would be great! In general, for the web platform tests, they would be great.
There is, IIRC an implementation of QuickCheck for javascript that could be made to work? I'm not sure if that quite qualifies as a fuzzer, but still.
It may even be possible to brute force...? Or, well, something like that, being a bit more clever about generating input based on the character sets taken from the spec.
It's not a priority for me right now (there are other things) but it would be good.


Batched matrix vector multiplication on Coral TPU

I am trying to move batched matrix multiplication to edge tpu. I have 3000 matrices of dimensions 500 * 500 and a vectors of dimensions 500 *1. I want to multiply matrix A_i with vector b_i and return the result. Matrices A_i are fixed while vectors will be variable.
Here are my questions:
1) Is my conversion code correct? I have followed the instructions at https://www.tensorflow.org/lite/performance/post_training_integer_quant
2) The conversion when num_matrices = 3000 takes too much time is there a way to speed this up?
3) When I have reduced dimensions of the problem I have managed to compile the model but it seems that not all operations are moved (quantizations reports that More than one subgraph is not supported). The result are faster on cpu
than tpu usb accelrator
4) When I change to the num_matrices=1000 the compiler takes a long time and returns internal compiler error.
System specification:
Edge TPU Compiler version 15.0.340273435 (I am using docker image for edgu-tpu compiler as described here https://github.com/tomassams/docker-edgetpu-compiler
)
tensorflow==2.4.1
tflite-runtime==2.5.0
numpy==1.19.5
Python 3.7.5
mac info:
System Version: macOS 10.15.7 (19H524)
Kernel Version: Darwin 19.6.0
I have written the following code for conversion:
MODEL_PATH = "matmul_ver1000.tflite"

dim_matrix = 500
num_matrices = 3000
a = np.random.random((num_matrices, dim_matrix, dim_matrix)).astype(np.float32)
b = np.random.random((num_matrices, dim_matrix)).astype(np.float32)

atf = tf.constant(a, dtype=tf.float32)
btf = tf.constant(b, dtype=tf.float32)
btf = tf.expand_dims(btf, axis=-1)

@tf.function
def matmul(a, b):
    return tf.matmul(a, b)

def _get_input_samples():
    atf = tf.constant(a, dtype=tf.float32)
    btf = tf.constant(b, dtype=tf.float32)
    yield [atf, btf]

def main():
    concrete_func = matmul.get_concrete_function(atf, btf)

    converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
    converter.optimizations = [tf.lite.Optimize.DEFAULT]

    converter.inference_input_type = tf.uint8
    converter.inference_output_type = tf.uint8
    converter.representative_dataset = _get_input_samples
    converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
    tflite_model = converter.convert()

    with open(MODEL_PATH, "wb") as fout:
        fout.write(tflite_model)
[matmul_ver100_edgetpu.log](https://github.com/google-coral/edgetpu/files/6465214/matmul_ver100_edgetpu.log)

However, the conversion takes a long time. I did the following experiments:

I have changed dimensions to dim_matrix = 100 and num_matrices =100 then the conversions finishes and was ble to compile the result with edge tpu compiler. Here are the results from edge compiler:

Model compiled successfully in 4499 ms.

Input model: matmul_ver100.tflite
Input size: 104.94KiB
Output model: matmul_ver100_edgetpu.tflite
Output size: 6.05MiB
On-chip memory used for caching model parameters: 4.93MiB
On-chip memory remaining for caching model parameters: 2.88MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 603
Operation log: matmul_ver100_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 402
Number of operations that will run on CPU: 201

Logfile from compiler is attached in matmul_ver100_edgetpu.log There is a line
QUANTIZE                       1          Operation is otherwise supported, but not mapped due to some unspecified limitation
QUANTIZE                       99         More than one subgraph is not supported
PACK                           1          More than one subgraph is not supported
matmul_ver100_edgetpu.log
I have run the inference on coral usb accelerator and results are better on cpu (15 ms) then tpu usb accelaror(183ms). Code is in inference.py attachment

I have changed num_matrices to 1000, the model converts but It returns internal compiler error:
Internal compiler error. Aborting!

Here is a code for inference:
import numpy as np
import time
import tflite_runtime.interpreter as tflite

MODEL_PATH_EDGE = "matmul_ver100_edgetpu.tflite"
MODEL_PATH = "matmul_ver100.tflite"

RUN_TPU = True
if RUN_TPU:
    import platform
    EDGETPU_SHARED_LIB = {
      'Linux': 'libedgetpu.so.1',
      'Darwin': 'libedgetpu.1.dylib',
      'Windows': 'edgetpu.dll'
    }[platform.system()]

    model_file, *device = MODEL_PATH_EDGE.split('@')

    delegate = [tflite.load_delegate(EDGETPU_SHARED_LIB)]
    interpreter = tflite.Interpreter(model_path=model_file, experimental_delegates=delegate)
else:
    interpreter = tflite.Interpreter(model_path=MODEL_PATH)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)

if input_details[0]['dtype'] == np.uint8:
    input_scale, input_zero_point = input_details[0]["quantization"]
    input_data = input_data / input_scale + input_zero_point
    input_data = input_data.astype(input_details[0]['dtype'])

interpreter.set_tensor(input_details[0]['index'], input_data)

input_shape = input_details[1]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
if input_details[1]['dtype'] == np.uint8:
    input_scale, input_zero_point = input_details[1]["quantization"]
    input_data = input_data / input_scale + input_zero_point
    input_data = input_data.astype(input_details[1]['dtype'])

interpreter.set_tensor(input_details[1]['index'], input_data)

tic = time.time()
interpreter.invoke()

output_data = interpreter.get_tensor(output_details[0]['index'])
toc = time.time()
print(f"Time elapsed = {toc - tic}")

total_time = 0.0
for _ in range(10):
    start = time.perf_counter()
    interpreter.invoke()
    inference_time = time.perf_counter() - start
    output_data = interpreter.get_tensor(output_details[0]['index'])
    total_time += inference_time
    print('%.1fms' % (inference_time * 1000))

print('Total time = %.1fms' % (total_time * 100))

It would be nice to get some answer here. :)
I advanced with this issue a bit so I am posting it here. The idea is that while matmul is not officially supported, the Dense layer can be used as a fake matrix vs vector multiplication. The main point is that matrices are constant so they should be put on tpu and not passed by set_tensor method. With the code below I managed to convert and compile the model for edge tpu. However, there seems that there is no memory for my case (num_matrices = 3000, dim_matrix = 500 gives ValueError: Cannot create a tensor proto whose content is larger than 2GB.). Anyway, I am posting it here in case anyone knows the right way how to do this.
I managed to compile the model with edgetpu_compiler test_matmul.tflite but got errors for operators of different rank
also memory used is not on chip. Here is a compiler output:
Edge TPU Compiler version 15.0.340273435
loc("Identity_int8"): error: two operands don't have the same rank
loc("Identity_int8"): error: two operands don't have the same rank

Model compiled successfully in 189 ms.

Input model: test_matmul.tflite
Input size: 977.47KiB
Output model: test_matmul_edgetpu.tflite
Output size: 1020.55KiB
On-chip memory used for caching model parameters: 0.00B
On-chip memory remaining for caching model parameters: 7.81MiB
Off-chip memory used for streaming uncached model parameters: 976.56KiB
Number of Edge TPU subgraphs: 1
Total number of operations: 1

Log file contains only
BATCH_MATMUL                   1          Mapped to Edge TPU
Here is a code for conversion of model to tflite.
import numpy as np
import tensorflow as tf
import tflite_runtime.interpreter as tflite

num_matrices = 100
dim_matrix = 100
A = np.random.random((num_matrices, dim_matrix, dim_matrix)).astype(np.float32)
B = np.random.random((num_matrices, dim_matrix)).astype(np.float32)

MODEL_PATH = 'test_matmul.tflite'

class MatMul(tf.keras.layers.Layer):
    def __init__(self):
        super(MatMul, self).__init__()

    def build(self, input_shape):
        self.w = tf.constant(A, dtype=tf.float32)

    def call(self, inputs):  # Defines the computation from inputs to outputs
        return tf.matmul(self.w, inputs)

# more inputs should be passed here for correct quantization
def _get_input_samples():
    b = np.expand_dims(B, axis=(0, -1))
    btf = tf.constant(b, dtype=tf.float32)
    yield [btf]

# create model
inputs = tf.keras.Input(shape=(num_matrices, dim_matrix, 1))
outputs = MatMul()(inputs)
model = tf.keras.Model(inputs=inputs, outputs=outputs)

# test model
input_data = np.expand_dims(B, axis=(0, -1))
input_tensor = tf.constant(input_data)
output = model(input_tensor)

# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
converter.representative_dataset = _get_input_samples
tflite_model = converter.convert()

# Save the model.
with open(MODEL_PATH, 'wb') as f:
  f.write(tflite_model)

# Run tflite model
interpreter = tflite.Interpreter(model_path=MODEL_PATH)
interpreter.allocate_tensors()
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()

input_shape = input_details[0]['shape']

# quantize input
if input_details[0]['dtype'] == np.int8:
    input_scale, input_zero_point = input_details[0]["quantization"]
    input_data = input_data / input_scale + input_zero_point
    input_data = input_data.astype(input_details[0]['dtype'])

interpreter.set_tensor(input_details[0]['index'], input_data)

interpreter.invoke()

# dequantize output
output_data = interpreter.get_tensor(output_details[0]['index'])
output_scale, output_zero_point = output_details[0]["quantization"]
output_real = (output_data - output_zero_point) * output_scale

@wwooster I have few questions to understand your end goal here.
so you want to perform matrix multiplication on 3000 matrices each 500x500 size ?
your input =[1, 3000,500,500]
output = [3000,500]
if so you have huge bottle neck due I/o operations between cpu and tpu. You are trying achieve 1000x more than I/o operations of regular models.
If the above assumption is wrong may I know what is your input, target operations on input and size of output.

Hello @Naveen-Dodda!
I would like to understand a few things here. My end goal is to perform batched matrix vs vector multiplication.
I have 3000 matrices of dimensions 500*500 stored in A of shape [3000, 500, 500]
Also 3000 vectors of dimensions 500 stored in B of shape [3000, 500, 1]
I want to multiply each matrix A[i, :, :] with corresponding vector B[i,:,:]
This can be achieved with tf.matmul operation,
output = tf.matmul(A, B)
output.shape = [3000, 500, 1]
What i would like is to move this computation to edgeTPU. I have written the script above to achieve this. Here are my problems:

The whole script seems to be overly complicated, is there an easier way how to do this?

When I compile the model with (num_matrices = 3000, dim_matrix = 500) I got ValueError:
Cannot create a tensor proto whose content is larger than 2GB)
I am ok with splitting computation on two dev boards.

When I set (num_matrices = 1500, dim_matrix = 500) I managed to convert the model to tflite, when compiling the model I got an error:

 loc("Identity_int8"): error: two operands don't have the same rank

Internal compiler error. Aborting!

Why is the model not able to compile and what is this error about?

When I further lower the dimensions to num_matrices =100 and dim_matrix =100
I manged to convert and compile the model with the following output:

loc("Identity_int8"): error: two operands don't have the same rank
loc("Identity_int8"): error: two operands don't have the same rank

Model compiled successfully in 223 ms.

Input model: test_matmul_100.tflite
Input size: 977.47KiB
Output model: test_matmul_100_edgetpu.tflite
Output size: 1020.55KiB
On-chip memory used for caching model parameters: 0.00B
On-chip memory remaining for caching model parameters: 7.81MiB
Off-chip memory used for streaming uncached model parameters: 976.56KiB
Number of Edge TPU subgraphs: 1
Total number of operations: 1
Operation log: test_matmul_100_edgetpu.log
See the operation log file for individual operation details.

Why is the memory on chip not used?
Thanks for your answers!

@wwooster
If it is a Deep learning model all the weights are fused and model would technically have fixed set of matrices on with which input is modified. But in your case you are not storing or fusing any specific matrix or weights in your model.
So technically edgetpu doesn't need cache memory. Yes off-chip memory is needed to off stream the output tensors.

@wwooster do you have any others questions regarding batched matrix vector multiplication on coral TPU


Show a line where max-line-length ends

In Atom editor, there was a function to enable a 30% transparent line at the max-line-length of the editor. I think that would be very useful when typing markdown or scripts too, to prevent beautifiers automated line wrap.

You can use editor.rulers for this purpose. See this StackOverflow question for more informations.
TL;DR
Add a vertical line at the 80th character with this code:
    "editor.rulers": [ 80 ]

Awesome, thanks!


[Bug] [connector][kafka] json deserialize exception log no content #3873

Purpose of this pull request
close #3873

Check list

[ ] Code changed are covered with tests, or it does not need tests for reason:
[ ] If any new Jar binary package adding in your PR, please add License Notice according
New License Guide
[ ] If necessary, please update the documentation to describe the new feature. https://github.com/apache/incubator-seatunnel/tree/dev/docs
[ ] If you are contributing the connector code, please check that the following files are updated:

Update change log that in connector document. For more details you can refer to connector-v2
Update plugin-mapping.properties and add new connector information in it
Update the pom file of seatunnel-dist

cc @hailin0  @Hisoka-X  PTAL, thanks.


Added support for a Japanese LLM model, rinna/youri-7b-chat

I added support for a Japanese LLM model, rinna/youri-7b-chat. The base architecture of the model is Llama2.
Note: The response from the LLM model is a bit shorter than I expected. It may be because of the strict system prompt message. If I use a more relaxed and short system prompt, the model response will become longer.

@yas-sim thank you for your contribution! It is great to see Japanese LLM support in chatbot notebook!
Could you please provide short description for model in README and notebook?

Ok. I have added a short description of Rinna-7b-chat model to README.md and the notebook. Please check them.


End-to-end multichannel wiener filters

[x] fix unit tests + regression tests
[x] update istft with torchaudio.istft
[ ] merge/update into dev branch
[ ] make inference jitable
[ ] test differentiable MWF

@aliutkus concerning the overall design of the Separator as I mentioned in the comments, we should focus on jitability but not on end-user experience (minimum RAM usage). Thus, I would allow users to still use numpy+norbert if they want minimum RAM usage.
This would greatly simplify the separator module (e.g. no loadings and checks in the forward path)

@aliutkus okay, I made some checks. Performance is great! It is significantly faster than before on cpu.
Now, one last thing: you forced the inference to run on torch.float64...

did you identify which tensor is affected the most with respect to numerical stability?
I tried with the default (float32), and the performance is only minimally affected (SDR was actually a tiny bit higher) Can we change this?
if not, lets make it at least an optional dtype parameter...

I checked tracebility of the model currently the model fails
/Users/faro/repositories/open-unmix-pytorch-dev/model.py:449: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for sample in range(nb_samples):
/Users/faro/repositories/open-unmix-pytorch-dev/model.py:452: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  while pos < nb_frames:
/Users/faro/repositories/open-unmix-pytorch-dev/model.py:453: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  t = torch.arange(pos, min(nb_frames, pos+batch_size))
/Users/faro/repositories/open-unmix-pytorch-dev/model.py:480: TracerWarning: Converting a tensor to a Python index might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  for sample in range(nb_samples)], dim=0)
RuntimeError: Only tensors, lists and tuples of tensors can be output from traced functions

the ranges can easily be fixed, but the last point requires to switch to tensors instead of dicts (it maybe be supported with in torch 1.6 though)

Next, I checked scriptability: this time we run into https://github.com/pytorch/pytorch/issues/28339 when indexing the nn.moduleList:
Traceback (most recent call last):
  File "test.py", line 104, in <module>
    torch.jit.script(model.Separator(targets))
  File "/Users/faro/repositories/open-unmix-pytorch-dev/env/lib/python3.7/site-packages/torch/jit/__init__.py", line 1261, in script
    return torch.jit._recursive.create_script_module(obj, torch.jit._recursive.infer_methods_to_compile)
  File "/Users/faro/repositories/open-unmix-pytorch-dev/env/lib/python3.7/site-packages/torch/jit/_recursive.py", line 305, in create_script_module
    return create_script_module_impl(nn_module, concrete_type, stubs_fn)
  File "/Users/faro/repositories/open-unmix-pytorch-dev/env/lib/python3.7/site-packages/torch/jit/_recursive.py", line 361, in create_script_module_impl
    create_methods_from_stubs(concrete_type, stubs)
  File "/Users/faro/repositories/open-unmix-pytorch-dev/env/lib/python3.7/site-packages/torch/jit/_recursive.py", line 279, in create_methods_from_stubs
    concrete_type._create_methods(defs, rcbs, defaults)
RuntimeError:
Only ModuleList, Sequential, and ModuleDict modules are subscriptable:
  File "/Users/faro/repositories/open-unmix-pytorch-dev/model.py", line 408

        for j, target in enumerate(self.targets):
            unmix_target = self.targets[target]
                           ~~~~~~~~~~~~~~~~~~~ <--- HERE

            # apply current model to get the source spectrogram

can be reproduces with
spec_model = torch.jit.script(model.OpenUnmix())
targets = {'vocals': spec_model}
torch.jit.script(model.Separator(targets))

I tested joint+end2end training in proof-of-concept branch (see https://github.com/sigsep/open-unmix-pytorch/pull/44)

@aliutkus again, I think going without dicts make everything easier

Hi, I was experimenting with the wiener function and ran into an issue when using it with mono audio stft (i.e. nb_channels = 1).
I added a small test on this branch which recreates it.
Here's a stack trace of the shape mismatch.
                    # apply it to the mixture
                    for i in range(nb_channels):
                        y[t, ..., j] = _mul_add(
                            gain[..., i, :],
                            x[t, ..., i, None, :],
>                           y[t, ..., j]
                        )
E                       RuntimeError: shape mismatch: value tensor of shape [10, 257, 2, 2] cannot be broadcast to indexing result of shape [10, 257, 1, 2]

filtering.py:351: RuntimeError
========================================================================================= short test summary info =========================================================================================
FAILED tests/test_filtering.py::test_wiener_shape[1] - RuntimeError: shape mismatch: value tensor of shape [10, 257, 2, 2] cannot be broadcast to indexing result of shape [10, 257, 1, 2]

@aliutkus was wiener_win_len=None when we used norbert?

@aliutkus please carefully check e1867694a405d8c9c62a0a32dd54f2d3689e9c6c for

are the eps replacements sensible?
anything else you would propose to simplify?

@aliutkus
expectation_maximization still uses batch_size, should be renamed.
Also the parameter is never passed there, so it is fixed to 200 even though  wiener_win_len is set differently. Furthermore default values of 200, 300, and 400 seem to be mixed.
What is the best value
Lastly, lets calculate this in seconds for test.py and eval.py

@aliutkus was wiener_win_len=None when we used norbert?

yes

@aliutkus please carefully check e186769 for
* are the eps replacements sensible?

* anything else you would propose to simplify?

concerning the eps, you just replaced it by a float with default value 1e-7, right ? I fear this does not make sense in the case double precision is chosen. Why don't you want to keep the eps of the computing dtype ?

concerning the eps, you just replaced it by a float with default value 1e-7, right ? I fear this does not make sense in the case double precision is chosen. Why don't you want to keep the eps of the computing dtype ?

finfo is not supported by ATen.

@cameronmaske

Hi, I was experimenting with the wiener function and ran into an issue when using it with mono audio stft (i.e. nb_channels = 1).

I added your test into our unit tests (b35193f) and I was able to reproduce the problem. I was unable to spot the place where this was caused
@aliutkus can you look into this?

@cameronmaske the mono case has been addressed in edca798

@aliutkus concerning variable re-naming:

use_softmask -> softmask (model.pyandfiltering.py`)
batch_size -> window_length (expectation_maximization in filtering.py)

@aliutkus maybe i missed this but concerning the residual model: is it important to have this after getting the softmask? If not, things could be made more elegant by removing this functionality from the Separator and instead allow users to pass their own residual model as a nn.Module. The default would just do the subtraction.
I actually think that appending this to the dict is a bit confusing. Another idea: Why not let the separator just output a tuple in any case (estimates, residual). I think it is useful in any case since users might be interested in the residual in nay case for assessing the remix capabilities...
does this make sense?

@faroit yes

@faroit
the user could always specify a module and name it rhe way he wants, like "residual"
the question we address here by definition is when we don't have a model for this. in that case, we do need to compute the residual the way we do, because it's the only way to have things working with EM
is it making sense?

@aliutkus concerning variable re-naming.. do you agree?
* `use_softmask` -> `softmask` (`model.py` and `filtering.py`)

@aliutkus I figured out this doesn't work as we have a softmask function. Ideas?

@aliutkus that was quite an effort 🍾

congrats


Tables lookups to assign variable

So the current table has a radial button to select a row, would it be possible to add a radial button to choose a column as well, and have a variable be assigned to the value at the intersection of the row and column?

On second thought, for the use case I'm thinking of, it's simpler to just snip an image of a table into a document cell and assign a value to a variable by inspection in a math cell below.


Local Runtime (Java) Memory Leak

Description
I am working on managed AI worker which spawns 715 entities, modify the position components and let SpatialOS update the data to runtime, every frame.
If I disable the worker (by disabling the worker connector before building it), and run the worker in Unity Editor, the memory usage stay stable at about 4GB memory working fine.
If I enable the worker connector and let it managed by the local runtime, the Java application consumes all my memory.
I've isolated the issue by testing the manged worker with/without the system that assign new value to Position.
Expected behaviour
The worker works normally whether it's managed/external.
(I expected workers running inside Unity should have more overhead)
Environment
i7-9700 / 32GB
Windows 10 1909 / Unity 2019.3.7f1
OpenJDK Runtime Environment (build 14.0.1+7)
OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)

I am working on managed AI worker which spawns 715 entities, modify the position components and let SpatialOS update the data to runtime, every frame.

So it turns out it's because a managed worker runs way faster then one running inside UnityEditor, so it generate enormous traffic toward the runtime.
After limit targetFrameRate to 60, the memory usage stays at about 4GB.

Unfortunately I have to reopen this since the memory usage will randomly grows, like, after staying at 4GB for 5 minutes, it suddenly goes up to 13GB, then after 2 minutes, it goes up to 15GB.

[DONE] Managed worker opening inspector (Issue reproduce in 8 mins, memory usage grows to 9GB)
I've reproduced the random memory leak, it happens when Inspector opened.

Another minor issue along with this:
Under heavy load Entity count in Inspector is being decreased overtime. The entity count decreased to 6XX, but shows 715 if I opens with Chrome at the time.

Does this happen with just the Inspector, or also Inspector V2?
When you mention CPU and memory usage, is this for the worker or the runtime process?
Note that sening an update on Position is special and rather heavy, as this will cause the runtime to re-evaluate all subscriptions made through QBI queries.

Does this happen with just the Inspector, or also Inspector V2?
I did not tested with InspectorV2, I've tried it multiple times though, but it's not as intuitive as V1 😅

When you mention CPU and memory usage, is this for the worker or the runtime process?
It's the java process. It always occupy a lot of CPU usage, I am thinking it may be related to what you said below.

Note that sening an update on Position is special and rather heavy, as this will cause the runtime to re-evaluate all subscriptions made through QBI queries.
Thank you, that's very valuable info!
Does this means I can give up Immediate updating of Inspector positions and use custom position component for performance?

Yes it is recommended to only update Position at a low rate, and use a custom component for the actual position of the entity.
Position is used for things like queries, the inspector, and for the initial prefab spawn if you use the default gameobject creator.

OK, thanks for the heads up.
I am in the progress to finish up a demo project and moving to cloud, so may not be able to reproduce the issue in near future, as we yet have a conclusion, I'll let this remain opened, will definitely report back if it happens again.


GUACAMOLE-1957: Support granular permissions management.

I've taken a stab at making the permissions management a bit more granular, allowing the four possible object-level permissions to be managed individually for connections, connection groups, and sharing profiles. As usual, I could use suggestions for how the GUI might be better set up to support this - I was originally going to try to create a table; however, going that route would require reworking the entire permissions management system, possibly implementing some sort of guac-group-table directive or something like that. So, I opted for a little simpler route.
Here's how it looks in practice:

Interesting!

Interesting!

"Interesting" can be such an ambiguous word. :rofl:

Interesting!

"Interesting" can be such an ambiguous word. 🤣

Heh - true. I think I like this approach, though. I'm curious how well it scales for larger numbers of connections.

Definitely a good point - I don't have all that many to test with in a test environment at the moment, so not sure how well the pagination works with it at the moment.


Cop for raise arguments doesn't account for a splat operator

Related to #552.
The following example causes an offense:
class MyException < StandardError
  def initialize(message, other_arg)
    super "#{message.inspect} -- #{other_arg.inspect}"
  end
end

args = ['message', 'other argument']

begin
  raise MyException.new(*args)
rescue MyException => exc
  puts exc.message
end

The offense here being  "Provide an exception class and message as arguments to raise."
I think the example is relatively self explanatory in what it's trying to achieve, but let me know if there's any questions.
Expected behavior
Rubocop should either not have a problem with use of the splat operator in this case, or detect if the array has more than one argument before raising the offense.
RuboCop version
$ rubocop -V
0.43.0 (using Parser <IP_ADDRESS>, running on ruby 2.3.0 x86_64-linux)

Rubocop should either not have a problem with use of the splat operator in this case, or detect if the array has more than one argument before raising the offense.

Yes. I think we should contend ourselves with not reporting when the splat operator is used, like in raise MyException.new(*args). Determining the size of the array is just too hard, generally speaking.


where is "pybullet-planning/pybullet_tools/ikfast/franka_panda"

I did not find the "pybullet-planning/pybullet_tools/ikfast/franka_panda". Where is the "pybullet-planning/pybullet_tools/ikfast/franka_panda".
Thank you!

Ah I forgot to add the submodule, thanks for pointing this out! This should be fixed with the latest commit. If you try cloning again with the --recurse flag, or pull and then run git submodule update --init --recursive, the pybullet-planning files should now be there.


feat: initiate Korean i18n effort

Part of https://github.com/huggingface/deep-rl-class/issues/370
Hello @simoninithomas ,
This PR marks the beginning of our effort to internationalize the Reinforcement Learning Course by adding Korean translations. The main goal of this initiative is to make our course more accessible to Korean-speaking users and expand our reach.
Here are the key changes included in this PR:

Created units/ko directory: This new directory will contain all our Korean translations. For this PR, it currently holds _toctree.yml with contents pointing towards the English version.

Added ko the language list of build_documentation.yml and build_pr_documentation.yml to enable views.

If you have any suggestions for improvements or spot any errors, please don't hesitate to point them out. Also, for anyone interested in contributing to this Korean i18n effort, your help would be much appreciated.
Looking forward to your feedback.
Best,
Won

Hey there👋 thanks for your work. As mentionned in Discord we don't support other languages for this course for now contrary to the transformer course.
However, what we can do is I can create a moon-ci-docs.huggingface.co link that will allow you to share to people who want to follow the course in Korean (and also see what the course looks like).
Currently there's an error in the Build PR documentation so I can't provide you this link (check the Failing error, from what I see is because you don't have a table of contents
Have a nice day 🤗


Split into separate subprojects for future multi-platform support

Splits the project into two modules, common and velocity. This will be convenient for bungee support in the future.
Shadow is still not configured correctly. I would like to avoid using compile since it's deprecated, but I haven't been able to exclude dependencies correctly.

Is there anything inherently velocity specific about VelocityEncoder/DecoderHandler?

Is there anything inherently velocity specific about VelocityEncoder/DecoderHandler?

Most of the current code comes from ViaVersion, you can see their bungee equivalent here https://github.com/ViaVersion/ViaVersion/blob/00b4c9b1f8220ae41363e7b5aae3dd0d6b1f2e12/bungee/src/main/java/us/myles/ViaVersion/bungee/handlers/BungeeDecodeHandler.java
https://github.com/ViaVersion/ViaVersion/blob/00b4c9b1f8220ae41363e7b5aae3dd0d6b1f2e12/bungee/src/main/java/us/myles/ViaVersion/bungee/handlers/BungeeEncodeHandler.java
It does seem very similar and part of the code could be common


Unexpected mismatch

first of all, thank you so much for this life saver. It really saved me probably days of debugging.
But there is one thing I can't understand. I'm passing all the blargg tests at this point but the 02-interrupts one doesn't get approved by your script. I get the following:
Mismatch in CPU state at line 151346:
MINE:	A:04 F:C--- B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFD PC:C2C0 PCMEM:05,C2,B9,C1
YOURS:	A:04 F:C--- B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFB PC:0050 PCMEM:3C,C9,00,00
However blargg tests print "passed". What can be the issue here? May it be because the rom itself has changed since your last upload?

I ran into this as well. I believe the logging is simply happening before the interrupts are checked. The next log at line 151347 is:
MINE:	A:05 F:C--- B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFB PC:0051 PCMEM:C9,00,00,00
So, likely it is logging PC at 0xC2C0, but the interrupt is going to 0x0050 and executing that instead.
Moving the logging around like this fixed the issue for me.

Thanks for your messages both. @cluoma to make sure I understand - you think that the example logs are correct, but users have to make sure to insert their logging at the correct point in their emulator? How do you think I could edit the docs to make this clearer?

I ran into a similar issue:
============== ERROR ==============

Mismatch in CPU state at line 151347:

MINE:	A:05 F:C--- B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFB PC:0051 PCMEM:C9,00,00,00
YOURS:	A:04 F:C--- B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFB PC:0050 PCMEM:3C,C9,00,00

The CPU state before this (at line 151346) was:

	A:04 F:10 B:01 C:00 D:C7 E:BA H:90 L:00 SP:DFFD PC:C2C0 PCMEM:05,C2,B9,C1

The last operation executed (in between lines 151346 and 151347) was:

	0x05 DEC B

Perhaps the problem is with this opcode, or with your interrupt handling?

It appears the the reference emulator will, all at once, set up the interrupt handler, and execute the first instruction of the handler.
So while my emulator was about to execute the first instruction of the handler (at 0x0050), the reference already has and is at 0x0051.
I think this should at least be documented.

@Ptival thanks for tracking that down - would you be up for making a PR with a README.md update that would have made this clearer for you?


fix: make resource group configurable in deploy.ps1

Description
make resource group configurable in deploy.ps1

Task Item
fixes #5773

Screenshots

Coverage remained the same at 53.422% when pulling b602dae2560ca85210946d9c081ef20f9515bc7c on qika/scriptUpdate into 61c944aba7cfe4f0b95120c5c2291b82468d3dc7 on main.


Enforcing that susceptibility is treated as a probability

Changelog:

Susceptibility is always treated as a probability (with domain [0,1]). Thus, the continuous susceptibility sampler now samples a user-parameterized beta distribution.
User parameter file now has more descriptive susceptibility and vaccine effect parameters
Utility functions to calculate beta distribution parameters from mean and variance
Step parameters can now use a list of values rather than defining a range to step through

Builds, compiles, runs (db init, single sim, verbose w/ simvis, batch sim)

This version successfully replicates expected analytical results when using discrete and susceptible susceptibility.


Check for partial downloads / corrupt archives

One time one user said on one reddit thread that something like this happened.
Seems sort of unlikely, but maybe considering adding some checks wouldn't be bad…

That's what the .bin.bak file is for but it would be nice. But, for this to happen Luma3DS would need to have checksums(sha,md5 ect) or he would need to host the checksum file him self

I was actually thinking of a really stupid check (like file size), if that matches and the archive can be opened and extracted, the chance of it being broken (but somehow working) are low enough.
An added check that can be done (as long as the file is small) is the MD5 sum.
Github uses AWS S3 to host files (and both official and hourlies use it) and S3 provides an "Etag" header which can be computed in different ways, but most of the time (unless multipart upload is involved, which only happens with big enough files) the ETag will be the MD5 of the uploaded file.
I think I'd make this second check optional since it 1. Might break in some cases 2. Take time to calculate the MD5 of the archive.

It shouldn't take that long to calculate MD5. But also I got a question, Now this is a bit off topic but due to where I am right now I can't visit many pages and must do it trough email, but anyway my question is how did you make a 3d model banner?!?!?!? There is a mass lack of 3d banners in 3ds homebrews because most people don't know how to do it. If you know of a tutoiral I missed during my google serch please link it.

I haven't used a tutorial (or found one when I searched). but it's sort of simple.
Every File Explorer has a OBJ -> CGFX converter. Just create your OBJ with model and texture mapped with your 3D modelling app of choice, then follow this path through EFE:

You need to get scale and rotation right as they're kinda messed up (you can use my luma model as reference).
Since there are some issues on how textures are handled (alpha in particular) and I'd love a command line tool for doing the conversion, I'm working on a CGFX toolchain myself, but it's in a very primitive stage right now.

Ok thanks once i get it working I'll put a tutorial on GBATemp or Reddit

On Jun 8, 2016, at 1:01 PM, ahcmaH<EMAIL_ADDRESS>wrote:
I haven't used a tutorial (or found one when I searched). but it's sort of simple.
Every File Explorer has a OBJ -> CGFX converter. Just create your OBJ with model and texture mapped with your 3D modelling app of choice, then follow this path through EFE:
You need to get scale and rotation right as they're kinda messed up (you can use my luma model as reference).
Since there are some issues on how textures are handled (alpha in particular) and I'd love a command line tool for doing the conversion, I'm working on a CGFX toolchain myself, but it's in a very primitive stage right now.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

So after a lot of fails  trying to make a 3d banner I gave up, but I decided to see what would happen if I did it with your model. So after opening it in Wings 3D > Adding Textures > Exporting > Converting in EFE this happened.

Its the same thing that has been happening to the other models and I can find a solution :/
I use EFE to convert and Wings 3D to make the models.

Sorry but I have no idea how that might have happened. I've only had issues with semitransparent textures so far, never something like that.
Have you been using luma.wings or luma.merged.wings ? The latter one has proper rotation/scale and has been all merged all in one mesh (and it's what I used to create the CGFX)

I have used the non merged one I'll try it rn

On Jun 9, 2016, at 4:52 PM, ahcmaH<EMAIL_ADDRESS>wrote:
Sorry but I have no idea how that might have happened. I've only had issues with semitransparent textures so far, never something like that.
Have you been using luma.wings or luma.merged.wings ? The latter one has proper rotation/scale and has been all merged all in one mesh (and it's what I used to create the CGFX)
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

To get back on topic, Integrity checks are in (both of them) with 245a597
Testing required.

I can test right now, I'll get back with the results

On Jun 10, 2016, at 5:54 AM, ahcmaH<EMAIL_ADDRESS>wrote:
To get back on topic, Integrity checks are in (both of them) with 245a597
Testing required.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

When I tried to build the new version it fails, this is the log. I got makerom and bannertool in my PATH the new ctrulib version and zlib.

 I've just updated to 3a10698 and it compiles fine.

The latest one

On Jun 10, 2016, at 3:39 PM, ahcmaH<EMAIL_ADDRESS>wrote:
What commit are you running? I've just updated to 3a10698 and it compiles fine.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

"The latest one" works for me, though. Do you really have it installed correctly and everything? The error you reported happens with the stable version (1.1.0).
Also, if you want to continue please query me on IRC (Freenode, Efnet) or use emails directly to<EMAIL_ADDRESS>This issue has been resolved with e450d80.

Odd I'll try latter

On Jun 11, 2016, at 3:41 AM, ahcmaH<EMAIL_ADDRESS>wrote:
"The latest one" works for me, though. Do you really have it installed correctly and everything? The error you reported happens with the stable version (1.1.0).
Also, if you want to continue please query me on IRC (Freenode, Efnet) or use emails directly to<EMAIL_ADDRESS>This issue has been resolved with e450d80.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.


feat: events page

This PR adds the Events page without integration with Google calendar (rebased copy of this PR
Preview

Thanks for all the work on this!


Cannot Reopen After Quitting on Mac

Describe the bug
Running on a Macbook pro with M2. Tried both app store and dmg installation.
When I checked the "Quit: Minimize to Tray" option and tries quitting with cmd+W, the localsend app is gone from my dock and is not showing up in the taskbar. Using activity monitor shows that the process is still running, so reopening does not work at all.
I was forced to kill the process and restart everything because of this.
To reproduce

Go to 'Settings'
checked the "Quit: Minimize to Tray" option
Quit or close the window
See error

Expected behavior
No error
Screenshots
No response
Desktop (please complete the following information)
OS: Sonoma 14.2.1 (23C71)
Smartphone (please complete the following information)
No response
Additional context
No response

I have the same problem.

same problem

I'll try to reproduce this issue tomorrow.


[Report page] [Bug] Make species names lowercase

Description
As per IDSEQ-2074, genus names should be uppercase and species names should be lowercase, but all taxon names were capitalized on the new report page.

Notes
text-transform: capitalize is applied on taxonNames by default to ensure genus rows are capitalized properly, which is then overridden with text-transform: none for species rows.
Tests

Verified that species names are in lowercase.

@gregdingle
It seems in the old report page, the non-specific rows weren't capitalized and there was a JIRA bug filed for that (IDSEQ-1749).

After using the method you suggested for capitalization, this is how the row now looks, which hopefully should be more natural casing 😄

Nice improvements!


Add solution for day 9

@julienrf do you think it would be better to use Vector instead of Array?
I use Iterator only to loop over data structures without the need to allocate intermediate lists or arrays. Should I replace with Lists?

My preference is that unless it makes a significant difference, it is fine to use the “non-lazy” collection types. That being said, if you don’t want to create collections for your intermediate results, you can use LazyList, which does not suffer from the issues of Iterator (although it has its own issues as well, such as the possibility of creating memory leaks if you are not vigilant…). I would also just use IndexedSeq instead of Array, unless you need mutability.


Unable to run Spoolman in Docker Container on MacOS

Describe the bug
If followed the Docker installation instructions for Spoolman, but when I try to start the container it continually reports the following errors:

I performed the "sudo chown -R 1000:1000 data" command, but that had no effect on the problem.
To Reproduce
Steps to reproduce the behavior:
Followed the instructions located here: https://github.com/Donkie/Spoolman?tab=readme-ov-file#docker-install
Expected behavior
Expected that Spoolman application would start.
Screenshots
If applicable, add screenshots to help explain your problem.
Spoolman Host (please complete the following information):

Installation type: Docker
Installed version: 0.19.3
OS and Distro: MacOS 12.7.5
Database type: SQLite

Desktop (please complete the following information):

OS: MacOS 12.7.5
Browser Chrome

Additional context
Add any other context about the problem here.

Updated macOS to 12.7.6, but this did not resolve the problem.

Oops, it appears that I closed this issue by mistake...

Would you mind printing the permissions using ls -n data/ to make sure the permissions are correct?
Another option might be to just use a docker volume, so it would turn the compose into something like:
services:
  spoolman:
    image: ghcr.io/donkie/spoolman:latest
    restart: unless-stopped
    volumes:
      # Mount the host machine's docker volume into the container's /home/app/.local/share/spoolman directory
      # "docker volume ls" to show the volume, and a "docker volume rm <spoolman vol from ls>" to permanently remove it
      - spoolman_data:/home/app/.local/share/spoolman
    ports:
      # Map the host machine's port 7912 to the container's port 8000
      - "7912:8000"
    environment:
      - TZ=Europe/Stockholm # Optional, defaults to UTC
volumes:
  spoolman_data:

keep in mind, you'll need to delete the volume if you want to permanently remove the data. docker compose down does not remove volumes.

I will do this, but it will be about two weeks until I am back at my
printer...
Thanks for the response.
On Sun, Sep 1, 2024, 05:00 RonanRaven @.***> wrote:

Would you mind printing the permissions using ls -n data/ to make sure
the permissions are correct?
Another option might be to just use a docker volume, so it would turn the
compose into something like:
services:
spoolman:
image: ghcr.io/donkie/spoolman:latest
restart: unless-stopped
volumes:
# Mount the host machine's docker volume into the container's /home/app/.local/share/spoolman directory
# "docker volume ls" to show the volume, and a "docker volume rm " to permanently remove it
- spoolman_data:/home/app/.local/share/spoolman
ports:
# Map the host machine's port 7912 to the container's port 8000
- "7912:8000"
environment:
- TZ=Europe/Stockholm # Optional, defaults to UTC
volumes:
spoolman_data:
keep in mind, you'll need to delete the volume if you want to permanently
remove the data. docker compose down does not remove volumes.
—
Reply to this email directly, view it on GitHub
https://github.com/Donkie/Spoolman/issues/441#issuecomment-2323139185,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGGPPWIZXIQUNN32QWHOGH3ZUJ7L3AVCNFSM6AAAAABMKE5CF6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGEZTSMJYGU
.
You are receiving this because you modified the open/close state.Message
ID: @.***>

@RonanRaven Here is a screenshot of the ls -n command output:

FYI: Spoolman now works in Docker when I use your updated docker-compose.yml per your instructions to use the docker volume...
Strange that I have to go through these extra steps in order to make it work... Is something not configured properly on my computer?

Screenshot of ls -na, in case it might be helpful...

im on latest macOS version and didn't have any issues I just inspected the data folder and manually set write permissions from the options on the bottom, however im unable to get any connection from my K1 printer running Fluidd to spoolman. I can open spool man in my browser just fine


Improved built_and_test.yml

What's done:

~removed setup gvisor runsc~
~reused docker client from testcontainers in save-orchestrator-common~ dockerClient from testcontainers should not be closed by us (testcontainers warns about it)
~extracted :save-orchestrator-common:check to a dedicated step~ extracted :save-orchestrator-common:build, :save-orchestrator:build and :save-sandbox:build to a dedicated step
moved downloading save-cli to a common step

May fix https://github.com/saveourtool/save-cloud/issues/1863

May fix #1863


Question about coloring behavior in UI

Hi there,
I've dug a bit through existing issues and the source code, but I'm not finding the answer to my question very easily, so I thought I'd just ask.
I started using the plugin yesterday, and I've noticed some behavior that seems odd to me but is likely intended.
Here's a habit block:
- ## Habits
	- DONE Meditate for 5 Minutes #[[Area: 🧠🧘🏻♂Mental Health]] #habit
	- LATER Complete a Workout #[[Area: Health]] #habit
	- DONE Read Daily Stoic #[[Area: Character Development]] #habit
	- LATER Publish an Atomic Essay #[[Area: Writing]] #habit

Notice that the habits I've checked off for today (8-16) are colored red and do not contain a number, while the habits I checked off for yesterday do contain a number, and the 1/d habits are colored green.
I take this to mean that you don't apply the final coloring/numbering to a cell until the full 24-hour day has elapsed. Is this correct?
If so, this was surprising behavior to me. I would have expected any habit which has met it's frequency/period goal to immediately be colored green (producing the subsequent "dopamine hit" from accomplishing a goal).
Thoughts?
Thanks again for a marvelous plugin!

Thanks, @mstine, for compliments.
I cannot exactly reproduce it.  Could you share your configuration and habit blocks for both today and yesterday?
Numbering and colouring should be immediate, though I think sometimes Logseq returns stale results to queries, if after checking any habit, numbers are not reflected, you may refresh or re-index.
Not 24-hour day, but habits are checked and coloured using a rolling window period. So for any given day if the frequency in the last n-days (7 days for a [w]eek including that day) is greater than or equal to the target, it is green, else red.
So within your first period, the day will be coloured in red until the target 5 has been surpassed within a week. In my opinion, it is the correct behaviour, though I'm open to suggestions.
Cheers

Sure! The below is after I reindexed a couple of times to ensure nothing was awry with my graph.
Your logic seems sound for the numbering and coloring, but it doesn't appear to work in my case.
Thanks again.
Config
{
  "disabled": false,
  "colors": [],
  "dateFormat": "MM-DD",
  "habits": {
    "Read Daily Stoic #[[Area: Character Development]]": {
      "period": "1/d",
      "order": 1
    },
    "Complete a Workout #[[Area: Health]]": {
      "period": "5/w",
      "order": 2
    },
    "Meditate for 5 Minutes #[[Area: 🧠🧘🏻♂Mental Health]]": {
      "period": "1/d",
      "order": 0
    },
    "Publish an Atomic Essay #[[Area: Writing]]": {
      "period": "5/w",
      "order": 3
    },
    "Brand new habit #[[Area: Finances]]": {
      "period": "1/d"
    }
  },
  "habitPattern": "^(LATER|DONE)?\\s*(?<habit>.*?)(?:| - (?:(?<int>\\d*?) times|(?<count>.*?)))$",
  "ignorePattern": "LATER .*",
  "hideStreak": false
}

Yesterday (August 15)
- ## Habits
	- DONE Meditate for 5 Minutes #[[Area: 🧠🧘🏻♂Mental Health]] #habit
	- DONE Complete a Workout #[[Area: Health]] #habit
	- DONE Read Daily Stoic #[[Area: Character Development]] #habit
	- DONE Publish an Atomic Essay #[[Area: Writing]] #habit

Today (August 16)
- ## Habits
	- DONE Meditate for 5 Minutes #[[Area: 🧠🧘🏻♂Mental Health]] #habit
	- DONE Complete a Workout #[[Area: Health]] #habit
	- DONE Read Daily Stoic #[[Area: Character Development]] #habit
	- DONE Publish an Atomic Essay #[[Area: Writing]] #habit

Latest Screenshot

Update just to show how things are progressing.
Today (August 17)
- ### Habits
	- DONE Meditate for 5 Minutes #[[Area: 🧠🧘🏻♂Mental Health]] #habit
	- DONE Complete a Workout #[[Area: Health]] #habit
	- DONE Read Daily Stoic #[[Area: Character Development]] #habit
	- LATER Publish an Atomic Essay #[[Area: Writing]] #habit

Latest Screenshot

Sorry for the late answer. I can't reproduce it. Is Logseq and the plugin updated to the latest version?

Did you re-index the graph? Could you try with a clean directory?

My apologies for the annoyance... when I checked for updates previously, nothing happened (network hiccup I guess), so I had assumed I was up to date. Apparently, I wasn't.
Updating to 0.8.1 fixed the problem (even without a reindex).

No problem. Happy to see your problem has been resolved.
Cheers.

Reopening this because the behavior keeps coming back (off and on), and I'm really not sure what to do.
I studied your source code to extract the query, and everything looks fine to me, but I haven't studied the coloring code enough to know what's happening to cause this:

The oddities here:

The two habits I checked off for today have gone back to the behavior where they are colored red with no number.
The habit (Complete a Workout) where I completed 5 days in a 7 day period (meeting the frequency/period goal) is red until the habit is completed on 8-19, and then green going forward until today where there is no color.
Also, for that same habit, there's a streak of 3? I do not understand that at all.

Let me know what data I can provide to help. Over the weekend, I did a lot of cleaning out of old cruft from when I moved from Obsidian to Logseq which I thought could corrupt the graph. I also disconnected, deleted the logseq folder, and reconnected. This briefly made things better, but then the confusing behavior returned within minutes.

Thank you for reopening.
While investigating, I've found a presentational bug, when there are < 14 days of habits, numbers and coloring are off in slice. I'm not sure whether your issue is triggered by this, but I think so.
Though, streak of 3 is curious. It should be 1. It seems to be a similar slicing error for the same edge case in success handling.
If I cannot solve it as quick as I want (tomorrow?), the release should have to wait next week.
Cheers

Apologies for the delayed response.
Since last commenting, I've continued tracking habits:
At present, even after reindexing the graph, the UI seems to be stuck in a strange state:

All habit tracking appears to stop on August 28th, even though I have continued tracking through today, September 2. (NOTE: I have changed my marker to #[[ATOMIC HABIT ⚛]], but that has worked as expected).
If I go back to August 1 and add habits:

The view gets very interesting indeed:

Curious.
Sorry, I couldn't find time. I'll be away on my honeymoon for a few weeks.

Hi there - I hope you had a nice honeymoon! I was wondering if you're going to find time to look at this one soon. If not, no worries. Just checking in. Thanks!

Thanks for reaching out and best wishes, @mstine. Sorry for late answer. I'm currently occupied with other things. I will try to find some time to look all the issues.


ARROW-2260: [C++][Plasma] Use Gflags for command-line parsing

The following patch ads Gflags support to plasma-store-server, leaves out the backtraces on invalid command-line options, and generally tries to make the error messages more useful in terms of nudging the user in the right direction.
Now, calling plasma-store-server alone prints plasma-store-server: please specify socket for incoming connections with -s, and the amount of memory (in bytes) to use with -m. Likewise with -h, although that's a happy coincidence (-h was already assigned to turn on hugepage support, and the command-line parsing errors on the missing mandatory options before getting to that).
Calling plasma-store-server -help produces:
plasma-store-server: Shared-memory server for Arrow data.
Usage:

  Flags from ../src/plasma/store.cc:
    -d (directory where to create the memory-backed file) type: string
      default: "/tmp"
    -e (endpoint for external storage service, where objects evicted from
      Plasma store can be written to, optional) type: string default: ""
    -h (whether to enable hugepage support) type: bool default: false
    -m (amount of memory in bytes to use for Plasma store, required)
      type: string default: ""
    -s (socket name where the Plasma store will listen for requests, required)
      type: string default: ""

  Flags from /tmp/gflags-20190105-85639-1w01usg/gflags-2.2.2/src/gflags.cc:
    -flagfile (load flags from file) type: string default: ""
    -fromenv (set flags from the environment [use 'export FLAGS_flag1=value'])
      type: string default: ""
    -tryfromenv (set flags from the environment if present) type: string
      default: ""
    -undefok (comma-separated list of flag names that it is okay to specify on
      the command line even if the program does not define a flag with that
      name.  IMPORTANT: flags in this list that have arguments MUST use the
      flag=value format) type: string default: ""

  Flags from /tmp/gflags-20190105-85639-1w01usg/gflags-2.2.2/src/gflags_completions.cc:
    -tab_completion_columns (Number of columns to use in output for tab
      completion) type: int32 default: 80
    -tab_completion_word (If non-empty, HandleCommandLineCompletions() will
      hijack the process and attempt to do bash-style command line flag
      completion on this value.) type: string default: ""

  Flags from /tmp/gflags-20190105-85639-1w01usg/gflags-2.2.2/src/gflags_reporting.cc:
    -help (show help on all flags [tip: all flags can have two dashes])
      type: bool default: false currently: true
    -helpfull (show help on all flags -- same as -help) type: bool
      default: false
    -helpmatch (show help on modules whose name contains the specified substr)
      type: string default: ""
    -helpon (show help on the modules named by this flag value) type: string
      default: ""
    -helppackage (show help on all modules in the main package) type: bool
      default: false
    -helpshort (show help on only the main module for this program) type: bool
      default: false
    -helpxml (produce an xml version of help) type: bool default: false
    -version (show version and build info and exit) type: bool default: false

which is rather verbose, but at least does the job.
Outstanding questions:

Is there a way to ask Gflags to suppress all that other stuff? Or do we ditch Gflags and go back to getopt() but with a handcrafted help option and message? Or is this good enough?
I'm assuming we want to be as backward compatible as possible? Otherwise I'd be tempted to move the non-optional command-line options to arguments, and maybe leave -h for help.
In the name of being as backward-compatible as possible, I'm exiting with -1 if the external store directory is not empty, but with 1 for other usage errors? Was that worth the tiny bit of complexity to preserve that behavior?
Which version define do I use to report plasma-store-server's version?
Any comments on the description of "Shared-memory server for Arrow data.", or is that good?

@chrish42 Thank you for the PR, I'll take a look now.  Note it looks like lint is failing due to formatting issues.  You need to run "make format" or "ninja format" to run clang-format (we are currently using version 8 I believe).

Okay, figured out how to run clang-format-8 on the code. (It feels like something that should definitely be easier, especially if it fails the build on the CI.) Let me know if you think anything is missing, because this looks done to me.

Hmm. The dev / lint failure is from arrow/rust/parquet/src/util/io.rs, which I didn't touch... 🤔

@chrish42 you can ignore the Rust lint failures until the Rust nightly linting issues are resolved

This seems like a strictly cleaner code, so I'll merge.  Thanks @chrish42

@chrish42 do you have a JIRA account setup, I'd like to assign the JIRA to you for book-keeping purposes.

@emkornfield I'm chrish42 on the Apache JIRA too. Thanks!


Upgrade to ETT 3.x.x

We're on version 2.4.0 currently. There are a few new features and bug fixes in 3.x.x already, including ones that affect the RO-Crate site.
Sub-tasks after an initial look locally:

[ ] "Related pages" dropdown no longer works on domain pages
[ ] Cards are highlighted in a dark colour on focus which affects contrast badly - tweak the colour or revert this to previous behaviour
[ ] Links in cards should be colour $dark not blue (as the whole card is the link)

The problems listed all derive from our override of section-navigation-tiles.html as the styling was significantly changed in ETT v3. We do still need to override it but we should be able to bring the file in line with v3.


Math Magicians - Calculator Events

Math Magicians - Calculator Events 😎
The following changes were implemented:

Created a new directory called logic inside the project ✅
Copied the files calculate.js and operate.js into the logic/ directory ✅
Installed big.js using npm ✅
Implemented click event handlers using the math logic from calculate.js and operate.js in the React component ✅
The Calculator is able to perform math operations✅
The operations and results are properly displayed ✅

Hi @franclobo,
Thank you for getting back to me so quickly. I removed the live demo because my first reviewer for Monday's milestone asked me to 😓. She suggested removing it per Microverse's new requirements and adding it after I have a deployed live demo.
Nevertheless, I'll add it again.
Once again, thank you for the kind feedback!

Hi @franclobo,
Thank you for getting back to me so quickly. I removed the live demo because my first reviewer for Monday's milestone asked me to 😓. She suggested removing it per Microverse's new requirements and adding it after I have a deployed live demo.
Nevertheless, I'll add it again.
Once again, thank you for the kind feedback!

Hi @mwismann the requirement is unless you haven't deployed yet the application you should replace the link with the "Comming soon" phase. 👍🏽


HOSTEDCP-2193: allow reuse of SP for e2e

What this PR does / why we need it:
This or adds a flag that we can use while executing the e2e in ci. The addition of this flag will allow us to pass in precreated SP and certs along with keyvault information that will then be used to populate the control plane managed identities field in the HC.
Which issue(s) this PR fixes (optional, use fixes #<issue_number>(, fixes #<issue_number>, ...) format, where issue_number might be a GitHub issue, or a Jira story:
Fixes #
Checklist

[ ] Subject and description added to both, commit and PR.
[ ] Relevant issues have been referenced.
[ ] This change includes docs.
[ ] This change includes unit tests.

/approve
we should follow up to clean up some of the CLI commands like the execute az cli command.

/lgtm

/test e2e-aws

/hold cancel