Client/Server API

Since we want thriftrw to better support HTTP (and presumably other transports that use Thrift message envelopes), let's get this discussion started.
Here's an inital API proposal for sync and async clients and servers.

For client-side support, we create a separate thriftrw-client Python package (probably a subdirectory under this project for now) that provides a thriftrw_client module with the following API:
import thriftrw
import requests
from thriftrw_client import Client

keyvalue = thriftrw.load('keyvalue.thrift')
client = Client(keyvalue.KeyValue, transport)
client.putValue("foo", keyvalue.Value("bar"))

try:
  client.getValue("foo")
except keyvalue.KeyNotFoundError:
  # ...

Where transport is any function that takes bytes to send the request and returns bytes of the response. This should suffice for synchronous clients and gevent based clients. We can provide default transport constructors like http_transport("http://.../thrift") will use httplib or something.
For asynchronous clients, we'll probably want to use thriftrw_client.tornado (and maybe later thriftrw_client.asyncio if needed) with the same interface except transport returns a future instead of the response as-is. These can be extras in the thriftrw-client's dependencies.
Open question: A generic Client class like that will have to do a bunch of introspection of the Thrift module at runtime. Do we want to do something similar to TChannel's client_for instead which generates the Client class specific to that service in one go?

For server-side, same story: A thriftrw-server package in a subdirectory with a thriftrw_server module that provides the following API:
import thriftrw
from thriftrw_server import Dispatcher

keyvalue = thriftrw.load('keyvalue.thrift')

dispatcher = Dispatcher(keyvalue.KeyValue)

@dispatcher.register
def getValue(self, key):
    # ...

@dispatcher.register
def putValue(self, key, value):
    # ...

# You can now do dispatcher(request_bytes) -> response_bytes
#
# So, if using Flask:

@app.route('/thrift')
def keyvalue():
    return dispatcher(request.data)

Same API for async servers except calling dispatcher returns a future. It'll plug into Tornado like:
class ThriftRequestHandler(tornado.web.RequestHandler):

    @gen.coroutine
    def post(self):
        response = yield disptacher(self.request.body)
        self.write(response)

Open question: What to name Dispatcher?

Other things to consider while looking at this API:

These APIs expect to do message envelope wrapping. We want that to be very clear. For non-TChannel users, that's the expected default. For TChannel, it's not.
Thrift has a TMultiplexedProtocol that wraps the real protocol (binary/json/whatever) except that it changes the method name in the message envelope to "$serviceName:$methodName". This allows the server to dispatch to multiple Thrift services on the same endpoint. How will this API be affected if we wanted to support that? The reason I want us to consider this early on is that with multiple Thrift services in the same file, it's definitely possible and common to multiplex Thrift services.

@blampe @breerly @junchaowu @kriskowal @prashantv @malandrew

Server side API may need a little bit more thought. This API won't support per request state to be passed in. Like if you're implementing an HTTP service you may want to inject information from the raw HTTP request (headers, tracing information, etc.) into the endpoint handler.
Actually that brings up the same issue in the client API. We want users to be able to inject per request information from the call to the transport.

Looking good to me. One thing:
Seems like you want to always return concurrent.futures.Future in the case of sync - any reason we can't do that?

@abhinav I'm not sure that we need to support request metadata in this library - after all, Apache Thrift suffers from the same issue. We'll be solving this in our RPC library.


[stable/home-assistant] some improvements

What this PR does / why we need it:
This pull request adds a few improvements

PodSecurityContext
Version bump
dnsPolicy for hostNetwork

Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] DCO signed
[x] Chart Version bumped
[x] Variables are documented in the README.md
[ ] title of the PR contains starts with chart name e.g. [stable/chart]

/ok-to-test

Nice changes @runningman84, thanks!
/lgtm


'Open ports on router’ option dissapears when there is no firewall zone called 'wan'

Hi
FIrst of all I have been configuring my BT Home Hub v5A router to be a wireless client that gives a NAT network on the 4 yellow ethernets and my own private network through the red socket. This all works fine but the reason I tell you this is that I renamed all of the zones so they were relevant to my setup hence why I renamed the default wan to nat_wwan
Open ports on router missing  (Network --> Firewall --> Traffic Rules)
when I rename the wan zone to nat_wwan the option 'Open ports on router’ option dissapears. I can get it to reappear by either renaming this zone back or creating a new zone with the name wan.
Also rules I have added do work even if this option is not here. (i.e. I added them before renaming the zone)
How it should look

What it looks like without the wan zone

So basically there is some sort of dependance on having a zone named wan, can this be fixed/removed.
My Setup

Firmare version  = OpenWrt 18.06.0 r7188-b0b5c64c22 / LuCI openwrt-18.06 branch (git-18.210.69179-6df9a57) - (18.06 Stable)
Router = BT Home Hub 5A (it is actually a Plusnet one)
if my configuration is of note I can supply that aswell

Thanks

This should work most probably with a named zone section, e.g. (untested!):
config zone 'wan'
        option name 'nat_wwan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'trm_wwan'

@dibdot
This is a workaround that you have to use the SSH to fix rather than using than using LuCI.
thanks anyway

@jow- thanks for the quick update
for reference I am running my OpenWRT as a wireless client (x2) which I named nat_wwan and bridge_wwan which are both wan connections so to keep things neat I used appropriate names and deleted the wan interface.


[ci] peg MiKTeX mirror

I learned tonight that the download link from the main MiKTeX site redirects to individual mirrors. I'm not sure what strategy it uses, but it seems to change pretty frequently. Here's an example from four requests executed within 15 seconds:

I think that pegging to a single mirror will reduce the frequency of errors like this downloading MiKTeX:

In theory that redirect mechanism they have is great because it means the download link should work whenever a particular mirror fails, but as we've seen it regularly routes us to a mirror that we can't successfully download from (for whatever reason). Since the r-package jobs already rely on the University of Illinois mirror being up (to install additional LaTeX packages), I'd like to propose that we should just go directly to that mirror for MiKTeX too.

ugh wait actually it seems that that mirror only hosts one artifact at a very very specific version number (https://ctan.math.illinois.edu/systems/win32/miktex/setup/windows-x64/)

which means this code will break every time a new version is pushed.
Ok I will close this and look for other strategies, sorry!


[BUG?] Startup failure OSError: [WinError 87] The parameter is incorrect

Description

When trying to start a Robyn app using the source code on the repo's README.md example, i get a massive error.
Traceback (most recent call last):
  File "C:\Users\Happy\OneDrive\Documents\Coding Projects\robyntests\app.py", line 9, in <module>
    app.start(port=5000)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\robyn\__init__.py", line 127, in start
    p.start()
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\reduction.py", line 63, in dump
    ForkingPickler(file, protocol, *args, **kwds).dump(obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\dill\_dill.py", line 620, in dump
    StockPickler.dump(self, obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 487, in dump
    self.save(obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 717, in save_reduce
    save(state)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\dill\_dill.py", line 1251, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 902, in save_tuple
    save(element)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 578, in save
    rv = reduce(self.proto)
TypeError: cannot pickle 'builtins.SocketHeld' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\spawn.py", line 102, in spawn_main
    source_process = _winapi.OpenProcess(
OSError: [WinError 87] The parameter is incorrect

Expected Behavior

Robyn runs I guess?
Screenshots

My code:

Environment:

Host OS: Windows
Browser:  Firefox
Python Version: 3.10.4

[Optional] Additional Context

I'm running my file using the generic python app.py

@justanotherbyte , can you please try running the app.py with the command:
python3 app.py --processes=1 --workers=1

And share the output with me?

Traceback (most recent call last):
  File "C:\Users\Happy\OneDrive\Documents\Coding Projects\robyntests\app.py", line 9, in <module>
    app.start(port=5000)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\robyn\__init__.py", line 127, in start
    p.start()
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\process.py", line 121, in start
    self._popen = self._Popen(self)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\context.py", line 327, in _Popen
    return Popen(process_obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\popen_spawn_win32.py", line 93, in __init__
    reduction.dump(process_obj, to_child)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\reduction.py", line 63, in dump
    ForkingPickler(file, protocol, *args, **kwds).dump(obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\dill\_dill.py", line 620, in dump
    StockPickler.dump(self, obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 487, in dump
    self.save(obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 603, in save
    self.save_reduce(obj=obj, *rv)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 717, in save_reduce
    save(state)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\dill\_dill.py", line 1251, in save_module_dict
    StockPickler.save_dict(pickler, obj)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 972, in save_dict
    self._batch_setitems(obj.items())
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 998, in _batch_setitems
    save(v)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 560, in save
    f(self, obj)  # Call unbound method with explicit self
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 902, in save_tuple
    save(element)
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\pickle.py", line 578, in save
    rv = reduce(self.proto)
TypeError: cannot pickle 'builtins.SocketHeld' object
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\Happy\AppData\Local\Programs\Python\Python310\lib\site-packages\multiprocess\spawn.py", line 102, in spawn_main
    source_process = _winapi.OpenProcess(
OSError: [WinError 87] The parameter is incorrect

same error

If it makes a difference, I am running the command like this: py app.py --processes 1 --workers 1 because I am on windows

@justanotherbyte , you're running it the right way. But this is not an expected behaviour.
I will have a look at this. Thank you!
Hopefully it should be fixed by the weekend.

Awesome thank you.

A potential fix looks like something here: https://stackoverflow.com/questions/36370724/python-passing-a-tcp-socket-object-to-a-multiprocessing-queue
Need to investigate more.

@justanotherbyte , can you check if it works on wsl?

Yes, it works on wsl

Does robyn only work on Linux?

Does robyn only work on Linux?

@justanotherbyte , it should work on all platforms. I am trying to set up a windows environment to fix this issue.

Blocked on this issue: https://github.com/sansyrox/robyn/issues/224

This has been fixed in #261. Thanks for the report.


Use mokito - HW 10.10.2016

Use mokito - commit

please add button to activity, check some flag or variable and in result show or hide button.
after, create test for that functionality

?


Field mappings

Add some extra fields and change some of the values for better compatibility with the S1 PowerQuery Language, no mayor changes on the pipeline or backend functionalities.
Regards

Sorry send pull request on the wrong repo, this was for S1 Power query, not S1QL


Student Grade Calculator

Project title
Student Grade Calculator
Project Description
Create a simple student grad calculator which takes subject marks of a student and it should print the grade with the remark, whether a student is passed or not.
Stack:
And in order to tick the check box just but x inside them for example - [x] like this. Please delete options that are not relevant.

[x] Html
[x] CSS
[x] JavaScript

Any GSSoC'22 contributor can contribute in this project.
I will assign the issue on the basis of first come first serve from tomorrow.

Can i work on this as a team??? @khushi-purwar Both of us are from GSSOC'22

No you cannot work in a team, because PR will be created by one member so he/she will get the points only.
I will suggest you, work individually. So there will be no conflict over marks at last.

Please assign this to me @khushi-purwar

Ok @deepu0201 you can work on this issue.


LND restarts with over 1GB channel.db on 32bit linux (RaspberryPi 4)

Background
LND is fails when the channel.db reaches the size of 1GB
Described previously here:
https://github.com/rootzoll/raspiblitz/issues/1778
Now  I had this repeatedly happen on two nodes and one has no leeway for further compaction.
Your environment

lnd: v0.11.1
Linux rpi4GB 4.19.118-v7l+ #1311 SMP Mon Apr 27 14:26:42 BST 2020 armv7l GNU/Linux
bitcoind v0.20.0
RaspiBlitz scripts behind Tor

Steps to reproduce
Run lnd and once the channel.db grows over 1GB it restarts repeatedly and reproducibly.
This last node seems to alway fail at  these log entries:
2020-11-30 17:36:57.455 [TRC] CHDB: Pruning nodes from graph with no open channels
2020-11-30 17:36:57.502 [INF] CHDB: Pruned unconnected node 038b731b984c3e9ef0d1710ffcd2ecdbddbf1d6097e900107d94ead7b7b1fd4956 from channel graph
2020-11-30 17:36:57.503 [INF] CHDB: Pruned unconnected node 03ba8210cea60dc7a98a0b6c87a71622d85a664ec203663dcebca09177de7a2563 from channel graph
2020-11-30 17:36:57.503 [INF] CHDB: Pruned unconnected node 021b00feca3ea07602cc12a44423735540d8212041864e9cc3c856a6171be88cc7 from channel graph
2020-11-30 17:36:57.504 [INF] CHDB: Pruned unconnected node 02151b861f4825eaed7c0e582c46e6af624ae61819eec7ae31f348e3dfa9615d6b from channel graph
2020-11-30 17:36:57.504 [INF] CHDB: Pruned unconnected node 021cd9c8d3cb0f934a6e446fc2a7db8e44c5a31b15d181d7d2f07128faf387dd27 from channel graph
2020-11-30 17:36:57.504 [INF] CHDB: Pruned 5 unconnected nodes from the channel graph

@Roasbeef has suggested that this might be the case after 2GB-s on a 32bit system, but seems it came earlier.
Working to migrate the project to 64bit OS: https://github.com/rootzoll/raspiblitz/issues/1199
And I will certainly do it for these nodes asap.
Do you think that switching the database between the 32 and 64 bit ARM architecture poses further risks?
Can be significant problem since the 32bit OS is the default recommendation for Raspberry Pi-s and many projects are based on that inlcuding: RaspiBlitz, RaspiBolt, myNode, Umbrel etc. (Nodl uses different SBC-s and aarch64 architecture from the beginning)

Do you have a breakdown of the file sizes in the entire .lnd directory?

Do you have a breakdown of the file sizes in the entire .lnd directory?

Sure:
$ sudo du -ah /mnt/hdd/lnd/
16K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/channel.backup
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/walletkit.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/chainnotifier.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/admin.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/invoices.macaroon
20K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/macaroons.db
2.0M	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/wallet.db
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/signer.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/invoice.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/router.macaroon
4.0K	/mnt/hdd/lnd/data/chain/bitcoin/mainnet/readonly.macaroon
2.1M	/mnt/hdd/lnd/data/chain/bitcoin/mainnet
2.1M	/mnt/hdd/lnd/data/chain/bitcoin
2.1M	/mnt/hdd/lnd/data/chain
9.3M	/mnt/hdd/lnd/data/watchtower/bitcoin/mainnet/watchtower.db
9.3M	/mnt/hdd/lnd/data/watchtower/bitcoin/mainnet
9.3M	/mnt/hdd/lnd/data/watchtower/bitcoin
4.0K	/mnt/hdd/lnd/data/watchtower/v3_onion_private_key
9.3M	/mnt/hdd/lnd/data/watchtower
21M	/mnt/hdd/lnd/data/graph/mainnet/sphinxreplay.db
1008M	/mnt/hdd/lnd/data/graph/mainnet/channel.db
33M	/mnt/hdd/lnd/data/graph/mainnet/wtclient.db
1.0G	/mnt/hdd/lnd/data/graph/mainnet/uncompacted.db
2.1G	/mnt/hdd/lnd/data/graph/mainnet
2.1G	/mnt/hdd/lnd/data/graph
2.1G	/mnt/hdd/lnd/data
4.0K	/mnt/hdd/lnd/v3_onion_private_key
4.0K	/mnt/hdd/lnd/lnd.conf
3.4M	/mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log
2.0M	/mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log.21554.gz
2.8M	/mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log.21553.gz
2.0M	/mnt/hdd/lnd/logs/bitcoin/mainnet/lnd.log.21552.gz
11M	/mnt/hdd/lnd/logs/bitcoin/mainnet
11M	/mnt/hdd/lnd/logs/bitcoin
11M	/mnt/hdd/lnd/logs
4.0K	/mnt/hdd/lnd/tls.key
4.0K	/mnt/hdd/lnd/lnd.conf.save
4.0K	/mnt/hdd/lnd/tls.cert
2.1G	/mnt/hdd/lnd/

Actually I left the backup up of the channel.db (uncompacted.db) in there too.
Now moved it out (just 5 mins ago - so no restarts since). Could that be part of the problem?

Will try to run the compaction there first.

Went on to migrate the 1GB channel.db to a 64bit ARM system. Looking good so far.
All channels and peers are online and can't see any errors in the LND.log.
Will give it some time now.

[signal SIGSEGV: segmentation violation code=0x1 addr=0x22aa0044 pc=0x2eb55c]

Oh oh, that sounds like an actual data corruption issue... Do you get the same error if you start that DB with lnd again?

Oh oh, that sounds like an actual data corruption issue... Do you get the same error if you start that DB with lnd again?

I could not start the 1GB+ channel.db on the 32bit system at all (LND restarted immediately), so went on migrating it to the aarch64 system and it is running there since without errors.
Could the chantools error be the same problem with bbolt and the size of the database?

Ah, okay. Yeah, it could be, what build/binary version of chantools did you run the command with? If there's no error when using 64bit ARM chantools, that's probably the problem.

I built chantools from the source at v0.5.1 on the 32bit system specifying the environment:
CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 make install
This has worked well before, but could only shrink the database less and less with repeated compactions. On the last one could only gain a few megabytes and it grew back in a couple of days again.
Testing chantools compactdb from the latest source on aarc64 resulted in no errors and went from:
1062508	to 1038376 bytes. LND started again without issues.
It seems that using a 64bit system does solve this issue, but can be a returning question since many projects are based off the default 32bit Raspbian.

That's good to know that a 32bit binary can result in a segfault for large DBs and that updating to 64bit helps.
But that doesn't solve the problem for existing users, I agree. We added the gc-canceled-invoices-on-startup flag that does some garbage collection. It's only on master now but will be in the next version. If you run with that set to true, then shutdown and compact again, do you see a big reduction in the size?

Just to confirm to test the garbage collection one should:

stop and run chantools compactdb
check channel.db size
build LND from source at the latest master (before lnd 0.12)
start the daemon with lnd --gc-canceled-invoices-on-startup (how long to wait for or look for some message in the lnd.log?)
check channel.db size
stop and run chantools compactdb
check the compacted.db size

Will the garbage collection and/or the compaction feature be on by default (happen on restarts) in lnd v0.12 or remains an option to  be run?

Yes, that's how I would try it as well. Though this might not do much if most data isn't from canceled invoices. It really depends on the usage of the node, but for now that is the only automatic garbage collection we've implemented (AFIK).
You should see a message in the logs mentioning the garbage collection. I don't recall the exact message.
Just to be clear: If you do this, you won't be able to downgrade the node to lnd v0.11.1-beta because of the database migrations it contains.
There is a new flag to run auto compaction on startup: --db.bolt.auto-compact
You might also want to set --db.bolt.auto-compact-min-age=0 to enable compaction on every restart and not just every week (which is the default). If you use that, you can skip the chantools compactdb calls and instead just restart lnd to achieve the same effect.

Just had a report from another active node (not under my control) experiencing the same immediate restarts with LND:
admin@raspberrypi:~ $ sudo du -h /mnt/hdd/lnd/data/graph/mainnet/channel.db
1.0G    /mnt/hdd/lnd/data/graph/mainnet/channel.db

and then with chantools (compiled for 32bit as described here: https://github.com/openoms/lightning-node-management/blob/master/LNDdatabaseCompaction.md):
bitcoin@raspberrypi:~/chantools$ chantools compactdb --sourcedb /mnt/hdd/lnd/data/graph/mainnet/channel.db \
>                 --destdb /mnt/hdd/lnd/data/graph/mainnet/compacted.db
2020-12-02 18:05:45.502 [INF] CHAN: chantools version v0.6.0 commit v0.6.0-1-gf82d78d
unexpected fault address 0x22a95044
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0x22a95044 pc=0x2eb55c]

He did not run chantools compactdb before.

That means the DB is too large to be opened with any 32bit process. If the user doesn't run a 64bit OS, they could copy it to a 64bit ARM device and run compaction there, with chantools compiled with 64bit. If compaction was never run, the benefit might be large to sufficiently postpone the problem again (until RaspiBlitz updates to 64bit in general).

Yes, it sounds clear that there is no solution to this on 32bit ARM and I have just been lucky to be able to run compaction when the issue presented the first time.
As he has no other ARM device advised to try the already functional 64bit RaspiBlitz version: https://github.com/rootzoll/raspiblitz/issues/1199#issuecomment-737305510. PR: https://github.com/rootzoll/raspiblitz/pull/1833
Thank you for the explanation about the upcoming features of LND:
--gc-canceled-invoices-on-startup and --db.bolt.auto-compact  will help to delay this issue for most and hopefully will give enough time for everyone to update to 64bit pre-emptively.

Another node reaching 1GB channel.db on 32bit ARM, just documenting the error message. Nothing left, but to migrate to 64 bit.
[INF] LTND: Version: 0.11.1-beta commit=v0.11.1-beta, build=production, logging=default
[INF] LTND: Active chain: Bitcoin (network=mainnet)
[INF] LTND: Opening the main database, this might take a few minutes...
[INF] LTND: Opening bbolt database, sync_freelist=true
[INF] CHDB: Checking for schema update: latest_version=17, db_version=17
[INF] LTND: Database now open (time_to_open=11.810386ms)!
[INF] RPCS: password gRPC proxy started at <IP_ADDRESS>:8080
[INF] RPCS: password RPC server listening on <IP_ADDRESS>:10009
[INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.
[ERR] LNWL: Failed to open database: cannot allocate memory

$ sudo du  /mnt/hdd/lnd/data/graph/mainnet/channel.db
1048544	/mnt/hdd/lnd/data/graph/mainnet/channel.db

$ lncli unlock
Input wallet password:
[lncli] rpc error: code = Unknown desc = cannot allocate memory

Update, as there is no other solution proposed here I am testing the 64bit Rasperry OS with the RaspiBlitz since opening the issue:
https://github.com/rootzoll/raspiblitz/issues/1199
The next (v1.7) SDcard release will be based on the 64bit base image and people can already download the RC1 from the dev branch: https://github.com/rootzoll/raspiblitz/tree/dev#downloading-the-software
If a 32bit system has this problem starting with the 64bit image is tested to solve the issue and LND will start again.

Closing this as the issue is with the 32-bit systems.

Hey @roasbeef sorry but I have to double check on this: Are you absolutely certain that you want to keep this closed and unfixed which would  effectively mean that lnd does not support 32 bit systems in the future (with all the (un)intended consequences that may come with such a decision)?

Two remarks:

for me moving the lnd data posed no issues, I migrated everything from an amd64 machine to armv7 (32 bit), bitcoind, electrumx and lnd data, and it all just worked; now I need to update the base system on my RPi to aarch64 before channels.db grows big enough... but there's still plenty of time. So I suppose there will be no issues if anyone decides to migrate, just make sure not to use the old data anywhere or nasty things will happen.
this size limitation is quite weird indeed, I'd expect 4Gb as it's the uint32 limit (or 2Gb for int32) but 1Gb is too little. Would be interesting to find the reason of these crashes. I only found this vague answer: https://github.com/boltdb/bolt/issues/535

I suppose it should either be fixed or documented on the main page that lnd explicitly doesn't support 32 bit systems and it's okay for it to crash as soon as the database reaches 1Gb (which isn't that much for a busy node), and you're left to pick up the pieces. Yes, the mainstream distros like Umbrel and Raspiblitz migrated to aarch64 but the very popular Raspbian (that I use) is still 32 bit by default and the 64 bit version is so unofficial/beta that I had to google it, there's no mention of it on the download page. I didn't even know it exists at all!

@renepickhardt there's nothing to resolve on our end which is the reason why this issue is closed. This is a matter of the architecture that lnd is running on, as well as the kernel related settings and how that interacts with the default database.
The reason why users run into this for 32-bit systems is entirely related to their kernel settings. bbolt uses memory mapping by default to map the entire database onto the virtual memory address space. Ignoring everything else, this means that the database size can grow up to 4 GB (not all physically mapped, but simply addressable). Once the DB size gets over 1 GB, bolt then attempts to re-size the memory map to attempt to double it to 2GB or so. At this point what happens next is dependent on the memory split of the user space vs the kernel. If kernel is set to (as an example) occupy 3 GB of space, with 1 GB for the user, then this operation will fail (example for 4GB of RAM some pis have less).
In terms of the kernel here, if users only want to run 32-bit systems, then I believe activating the PAE extensions for the "high memory" kernel operating mode can help here. The issues users that pack everything running on a single pi (certainly not advised if you want reliability, but again you get what you pay for with a rasp pi), is that other processes (bitcoind, etc) are also competing for the address space. Depending on swap settings and addressable physical RAM, not everything may be able to fit.
I wager that only very old or very large nodes running on pi's etc (that haven't examined their kernel settings at all), need to deal with this. For larger nodes, if a node is that larger then it would behove the user to switch to more reliable hardware as pi's notoriously can run into hardware failure issues.
In practice, if a user ever detects this, then it's likely due to the fact that they may have never compacted their database. bolt doesn't actually reclaim the space when things are deleted, instead it keeps it all around on disk, putting the free pages in a free list for new DB operations. Users that run into this can usually just compact their database and the issue goes away. The latest versions of lnd (master to be made into 0.14) are also better about deleting state they not longer need, which results in smaller database sizes.
Beyond that, lnd 0.14 will ship with a newly optimized etcd backend, as well as initial support for postgres. For users seeking to operate a more reliable set up, both those options are certainly better than potentially storing all the data on an SD card.

To provide a bit more detail as to why the operation can fail (lets assume 4 GB of RAM, DB is 2 GB at this point), see this method: https://github.com/etcd-io/bbolt/blob/master/node.go#L517
What happens is the bolt needs to copy the entire database into heap memory temporarily to ensure that once it unmaps, then maps the database, the inodes, etc aren't pointing to stale memory. I think this is the error most people are running into: when then DB needs to double in size initially (and any 1 GB increments beyond that) there simply isn't enough addressable physical memory.

One attempt to mitigate this somewhat in the past was trying to set an initial memory map of the largest DB a 32-bit system can handle: https://github.com/btcsuite/btcwallet/pull/697
This would mean that the DB never needs to be copied over as it would never need to be remapped. IIRC we fell short on testing on that, and also just generally concluded that in 2021 maybe the effort to make things slightly safer for 32-bit system may not have been worth it. Since then IIRC, we entertained removing the compiled 32-bit binaries, but then decided that for smaller nodes (as they should be on a pi) regular compaction resolves the issue in practice.


PanicException: assertion left == right failed

Hi and thanks again for maintaining this open source solver :smiley: I noticed the following exception thrown by Clarabel on the QUADCMPC1 problem of the free-for-all QP test set:
$ python free_for_all.py run --solver clarabel --problem QUADCMPC1
results_file='src/free_for_all_qpbenchmark/results/free_for_all.csv'
[2024-07-31 14:30:39,925] [info] Loading existing results from src/free_for_all_qpbenchmark/results/free_for_all.csv (results.py:78)
[2024-07-31 14:30:39,953] [info] Solving QUADCMPC1 by clarabel with default settings... (run.py:108)
thread '<unnamed>' panicked at src/solver/implementations/default/solver.rs:56:9:
assertion `left == right` failed
  left: 1536
 right: 1408
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "src/free_for_all_qpbenchmark/free_for_all.py", line 231, in <module>
    main(test_set_path=os.path.abspath(__file__))
  File ".micromamba/envs/qpbenchmark/lib/python3.12/site-packages/qpbenchmark/benchmark.py", line 268, in main
    run(
  File ".micromamba/envs/qpbenchmark/lib/python3.12/site-packages/qpbenchmark/run.py", line 113, in run
    solution, runtime = time_solve_problem(
                        ^^^^^^^^^^^^^^^^^^^
  File ".micromamba/envs/qpbenchmark/lib/python3.12/site-packages/qpbenchmark/utils.py", line 129, in time_solve_problem
    solution = qpsolvers.solve_problem(problem, solver=solver, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".micromamba/envs/qpbenchmark/lib/python3.12/site-packages/qpsolvers/solve_problem.py", line 81, in solve_problem
    return solve_function[solver](problem, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".micromamba/envs/qpbenchmark/lib/python3.12/site-packages/qpsolvers/solvers/clarabel_.py", line 118, in clarabel_solve_problem
    solver = clarabel.DefaultSolver(P, q, A_stack, b_stack, cones, settings)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: assertion `left == right` failed

I checked quickly that A_stack.shape=(2432, 768) and b_stack.shape=(2432,) (could be "left" and "right"?). Other solvers like OSQP solve the problem successfully.
Sorry I don't have the time right now to make a reproducer that does not depend on the benchmarking library, but the exception should be reproducible (after conda/PyPI installing qpbenchmark) by:
$ python free_for_all.py run --solver clarabel --problem QUADCMPC1

Finally I noticed that Clarabel 0.7.0 does not raise the exception, while Clarabel 0.8.0 does.
Hoping this helps!

It looks like that error is thrown when there is a mismatch between the dimension of b and the total of the dimensions in the cones that is passed to the solver.   What does cones look like?

The cones look like this:
A.shape=(384, 768)
G.shape=(2048, 768)
A_stack.shape=(2432, 768)
b_stack.shape=(2432,)
cones=[ZeroConeT(384), NonnegativeConeT(2048)]

Here is the context for reference.

The cones appear to be compatible, although the total dimension (2432) is lower than both sides of the assertion that failed.
In version 0.8.0 we reimplemented the preprocessing step that eliminates infinite bounds, so perhaps an error was introduced there.   Does b_stack have a lot of very large entries.   Specifically, does it have 1000 entries with values >= 1e20?

Yes it does :dart:
In [3]: (b_stack >= 1e20).sum()
Out[3]: 1024


Completed functionalities of the authentication modules

Complete UI and navigation for authentication modules
Completed Firebase integration
Introducing Hilt for injection
Adding separate composables for modularity and clean code
Adding R8 and proguard for code optimization

Solved conflicts

Merging auth into main branch then deleting


when python mnist_sslvae.py

in shell:"python mnist_sslvae.py"
i choose " from models import LadderDeepGenerativeModel
models += [LadderDeepGenerativeModel([784, n_labels, [32, 16, 8], [128, 128, 128]])]”
i met the problem:
"Traceback (most recent call last):
File "mnist_sslvae.py", line 117, in
total_loss += J_alpha.data[0]
IndexError: invalid index of a 0-dim tensor. Use tensor.item() to convert a 0-dim tensor to a Python number"

i wonder that
need i change every data[0] to item()?any other solution?

You need to downgrade to PyTorch 0.4.0.
Please stop creating issues for problems not related to the codebase. You can find these solutions on sites like stackoverflow.


Batch call link

The Batch call link seems to be missing:
https://docs.microsoft.com/en-us/rest/api/vsts/wit/batch/update.yml?view=vsts-rest-4.1
404 NOT FOUND

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 5aa4c957-4526-9043-cb91-5e0bee7734ea
Version Independent ID: 450b9f73-dbbe-0f0f-c46f-079d33a92884
Content: Work item tracking REST API for Visual Studio Team Services (VSTS) - Visual Studio Team Services REST API
Content Source: docs-ref-conceptual/4.1/wit/index.md
Service: vsts
Product: devops
GitHub Login: @wnjenkin
Microsoft Alias: elbatk

@jessehouwing thanks for your feedback. We're investigating this issue

@jessehouwing the batch call api can be found here - https://docs.microsoft.com/en-us/rest/api/vsts/wit/WorkItemBatchUpdate?view=vsts-rest-4.1


Referencing a Table (tablex)

When trying to add the <reference-label> to the table and then @referencing it, I get the error "cannot reference sequence".

I've also tried putting everything into a #figure(), but that just leads to weird issues:

(there's also a completely empty page btwn the "Appendix" heading and the table, which is only there when wrapping it in #figure(). I've set the repeat-heading setting to true, which worked well in the non-#figure()-wrapped version.)

you might want to submit this issue to https://github.com/PgBiel/typst-tablex

There is already this issue at tablex's repository: https://github.com/PgBiel/typst-tablex/issues/7
I'm going to close this here.


Scan the content after git commit

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Whether to support scanning the content of this submission after git submission, which is very helpful for code review
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

usually we do scan the git commit BEFORE it is made, but not AFTER.
you can achieve this using idea like this:

troggle the checkbox "perform code analysis" and then you click commit, it will pump up a hint like the screenshot shown.
click the blue [Review] button then the codes will be analysed.

usually we do scan the git commit BEFORE it is made, but not AFTER.
you can achieve this using idea like this:

troggle the checkbox "perform code analysis" and then you click commit, it will pump up a hint like the screenshot shown.
click the blue [Review] button then the codes will be analysed.

Scanning after submission is sometimes very useful, for example, when looking at the branch code submitted by someone else, I want to see if the code he submitted this time meets the specifications

流程机制不放在idea插件体系中，可以考虑自己集成在CI体系中


install terraform in test coverage job

@iskey kindly merge this whenever possible since the other team members also need it. This PR should fix all other failing PRs too.

@iskey kindly merge this whenever possible since the other team members also need it. This PR should fix all other failing PRs too.

ok，it looks good.


tasks

[x] Turn on GitHub Pages
[x] Outline my portfolio
[x] Introduce myself to the world

vdsdsfewfw


Toggle Torch and more?

like wifi-toggle
or maybe ability to add any action (similar to https://play.google.com/store/apps/details?id=com.sika524.android.quickshortcut )

There is already such ability. Go to Menu -> List Selection -> activity -> back to list -> long tap on wanted activity -> add


Generating bindings for Vulkan 1.0.25 fails

I don't know enough about the format used in the vk.xml file, however it seems that generating bindings for the latest revision of the Vulkan spec fails at WriteMemberArray because of being unable to find the 'len' member for a function pointer, specifically for pAcquireTimeoutMilliseconds: https://github.com/KhronosGroup/Vulkan-Docs/blob/1.0/src/spec/vk.xml#L2246

I think this is because pAcquireTimeoutMilliseconds is meant to be (from what I can tell), a const uint32_t* but not an array, however, the Generator is unable to determine that and ends up calling WriteMemberArray.

It should be now fully fixed by https://github.com/mono/VulkanSharp/commit/ddba19b025184f338160174a05cdad8d12832f90


[#151] Precedents data structure

Created structure for precedent data
This data can also be outputted as JSON

Precedent data structure:
{
"AZ-123456":{
              "facts": [(0,"some fact"),(10,"another fact with different category")...]
              "facts_vector": [0,1,1,1,1,1,0,0,0,1,0]
              "outcomes": [(0,"some fact"),(10,"another fact with different category")...]
              "outcomes_vector": [0,1,1,1,1,1,0,0,0,1,0]
              "lease_terminated": None
            }
"AZ-1234567":{
              "facts": [(0,"some fact"),(10,"another fact with different category")...]
              "facts_vector": [0,1,1,1,1,1,0,0,0,1,0]
              "outcomes": [(0,"some fact"),(10,"another fact with different category")...]
              "outcomes_vector": [0,1,1,1,1,1,0,0,0,1,0]
              "lease_terminated": None
            }
}

I disagree.
"AZ-123456":{
"facts": [(0,"some fact"),(10,"another fact with different category")...]
"facts_vector": [0,1,1,1,1,1,0,0,0,1,0]
"outcomes": [(0,"some fact"),(10,"another fact with different category")...]
"outcomes_vector": [0,1,1,1,1,1,0,0,0,1,0]
"lease_terminated": None --> DON'T NEED THIS
}

Save as pickle too please. More efficient to load

@Taimoorrana1 it's only a suggested template. We don't actually need that value because we should know it's index in the array

Though it may correspond to multiple clusters I think it's correct like this


Filter for NULL values

Is your feature request related to a problem? Please describe.
I have a datetime field (e.g. deactivated_at) which can be NULL. I'd like to filter for all records which have a NULL value there. Or the opposite: have a value set there.
Describe the solution you'd like
Maybe just another operator which supports is_nil.
Or is there another good way to do it?

That seems to be missing, indeed. Since the operator names are potentially user-facing, I'd probably name them empty and not_empty, rather than is_nil or is_null.
Since the filter value is required, you cannot set nil as a value at the moment.


Error 404 when using ctx.send()

@slash.slash(name='economy', guild_ids=guild_ids, description="Shows the economic data of a nation.", options=[manage_commands.create_option(name='nation', description='This is the nation you wish to see the economic data for...', option_type=3, required=True)])
async def _economy(ctx, nation):
    switchSheet('GDP Growth')
    print(nation)

    countryNum = findNation(nation)
    print(countryNum)

    if countryNum > 0:
        ecoEmbed = discord.Embed(title="Economy Info",
                                 description="This is a summary of {0}'s Economy Information".format(
                                     str(sheet.get('A{0}'.format(countryNum))).replace("[['", "").replace(
                                         "']]", "")),
                                 colour=discord.Colour(int(colors.rgb2hex([get_effective_format(sheet, 'A{0}'.format(countryNum)).backgroundColor.red,
                                                                           get_effective_format(sheet, 'A{0}'.format(countryNum)).backgroundColor.green,
                                                                           get_effective_format(sheet, 'A{0}'.format(countryNum)).backgroundColor.blue]).replace('#', '0x'), 0)))
        ecoEmbed.add_field(name="GDP:",
                           value=str(sheet.get('AX{0}'.format(countryNum))).replace("[['", "").replace(
                               "']]", ""), inline=False)
        ecoEmbed.add_field(name="GDP Growth:",
                           value=str(sheet.get('B{0}'.format(countryNum))).replace("[['", "").replace("']]",
                                                                                ""),
                           inline=False)
        ecoEmbed.add_field(name="Growth Value:",
                           value=str(sheet.get('D{0}'.format(countryNum))).replace("[['", "").replace("']]",
                                                                                ""),
                           inline=False)
        ecoEmbed.add_field(name="GDP Next Year:",
                           value=str(sheet.get('AZ{0}'.format(countryNum))).replace("[['", "").replace(
                               "']]", ""), inline=False)
        ecoEmbed.add_field(name="Consumer Confidence:",
                           value=str(sheet.get('AU{0}'.format(countryNum))).replace("[['", "").replace(
                               "']]", ""), inline=False)

        await ctx.send(embed=ecoEmbed)
    else:
        print('Invalid Nation...')

This code fails when using ctx.send() and leaves an error 404 (see attached) & responds to the user of the slash command with 'Interaction Failed' however when using ctx.channel.send() it sends the intended response with no error however sends the 'Interaction Failed' message.
With "ctx.send()":
With "ctx.send()":

With "ctx.channel.send()":

for the interaction failed when using ctx.channel.send() thats because you are technically not responding to the interaction, thus it "fails", you always needs at least 1 ctx.send() to prevent the interaction failed message.
but i dont know why ctx.send() is failing though

for the interaction failed when using ctx.channel.send() thats because you are technically not responding to the interaction, thus it "fails", you always needs at least 1 ctx.send() to prevent the interaction failed message.
but i dont know why ctx.send() is failing though

Yeah I knew about the channel.send thing however it was used to test if the issue was with the declaration of the ctx and I found it interesting that it didn't refuse to send when using that

after another test I've found that it happens even outside of the if statement.

Do you have another instance of this bot running (accidentally or something)?
What is your discord.py and discord-py-slash-command version?

Do you have another instance of this bot running (accidentally or something)?
What is your discord.py and discord-py-slash-command version?

Discord.py = 1.7.3
Discord.py (slash command) = 2.4.1
and it is 100% only running one instance and other commands using ctx.send() are working

Try adding await ctx.defer() at the top of the code?

Try adding await ctx.defer() at the top of the code?

I honestly feel very stupid now, In all my usage of ctx.defer() I have forgotten the await at the beginning, thank you very much lmao I appreciate it


zipkin web cannot search the longest duration between two time

in zipkin web ,the search result is sorted by timestamp ,and then sorted on that result ,
we usually want to find the longest trace between some time,but the result from searching is not our wanted.

The duration query was made for this type of search. Can you use this for
now? (understood that you need to know the value you want to constrain
before you search)

i think the ui can increase some function，such as find the real longest trace in some time , and after search, the search condition is lost ,and it should not refresh all the page ,just refresh the search result,I think it is not friendly to users

i think the ui can increase some function，such as find the real longest
trace in some time , and after search, the search condition is lost ,and it
should not refresh all the page ,just refresh the search result,I think it
is not friendly to users

The UI already has an option to sort results by duration.
The problem you are referring to is the backend. If I understand you right,
you want the server to return you the longest duration traces constrained
by other parameters. For example, if there were 2 million traces between
today and yesterday, return the 10 that had the longest duration. Correct?

@adriancole yeah,just that meaning

So, the biggest change to support this would be related to indexing. For example, it is not realistic to continually search through a very large amount of traces even if a small amount could be done without noticing. It also is unlikely to work client-side for many sites as the data is too big.
We've some similar work in place to support the duration query, but it is a big deal feature and would  require different work for each data store (ex the solution for MySQL will be different than Cassandra, which is different than Elasticsearch).
So, the main concern is that we would need some relatively large effort to accomplish this feature.

last time,I submit a issue about the brave on  https://github.com/openzipkin/brave/issues/355,
in the  class "InheritableServerClientAndLocalSpanState"
i modify the currentServerSpan and currentLocalSpan like this:
`
private final InheritableThreadLocal<Deque> currentServerSpan =
new InheritableThreadLocal<Deque>() {
@Override
protected Deque initialValue() {
return  new LinkedBlockingDeque();
}
};
private final InheritableThreadLocal<Deque<Span>> currentClientSpan =
        new InheritableThreadLocal<Deque<Span>>() {
            @Override
            protected Deque<Span> initialValue() {
                return  new LinkedBlockingDeque<Span>();
            }
        };

private final InheritableThreadLocal<Deque<Span>> currentLocalSpan =
        new InheritableThreadLocal<Deque<Span>>() {
            @Override
            protected Deque<Span> initialValue() {
                return  new LinkedBlockingDeque<Span>();
            }
        };`

it can solve the problem that diffrent dubbo invocation in the same thread,i hope it would be added into  next brave edition,

@zengzonghou can you move the above comment to an issue on brave? I don't know which and this is unrelated to the duration query. It will help us make sure we solve each problem independently


aws-c-common: 0.3.11 -> 0.4.63

Automatic update generated by nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/aws-c-common/versions.
meta.description for aws-c-common is: "AWS SDK for C common core"
meta.homepage for aws-c-common is: "https://github.com/awslabs/aws-c-common"
meta.changelog for aws-c-common is: ""
Updates performed

Version update

To inspect upstream changes

Release on GitHub

Compare changes on GitHub

Impact

Checks done (click to expand)

built on NixOS

The tests defined in passthru.tests, if any, passed

0 of 0 passed binary check by having a zero exit code.

0 of 0 passed binary check by having the new version present in output.

found 0.4.59 with grep in /nix/store/dgjd0jhk31fvifsn584qr6cwch4kdcqy-aws-c-common-0.4.59

directory tree listing: https://gist.github.com/d0b637b1bb33841e53e87948e33d16f1

du listing: https://gist.github.com/b8f46bbab99931f2b0f6ae55756dc120

Rebuild report (if merged into master) (click to expand)

211 total rebuild path(s)

58 package rebuild(s)

58 x86_64-linux rebuild(s)
52 i686-linux rebuild(s)
50 x86_64-darwin rebuild(s)
51 aarch64-linux rebuild(s)

First fifty rebuilds by attrpath
aws-c-common
aws-c-event-stream
aws-sdk-cpp
bundix
cabal2nix
cachix
common-updater-scripts
crystal2nix
dep2nix
disnix
disnixos
fusionInventory
go2nix
haskellPackages.cachix
haskellPackages.hercules-ci-agent
haskellPackages.hocker
haskellPackages.nix-paths
hercules-ci-agent
hydra-unstable
lispPackages.quicklisp-to-nix
lispPackages.quicklisp-to-nix-system-info
nix
nix-bundle
nix-direnv
nix-doc
nix-du
nix-exec
nix-index
nix-pin
nix-plugins
nix-prefetch
nix-prefetch-bzr
nix-prefetch-cvs
nix-prefetch-docker
nix-prefetch-git
nix-prefetch-hg
nix-prefetch-scripts
nix-prefetch-svn
nix-serve
nix-update
nix-update-source
nixFlakes
nixStable
nixUnstable
nixos-generators
nixos-shell
nixpkgs-review
nixui
python37Packages.nix-kernel
python37Packages.nixpkgs

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/dgjd0jhk31fvifsn584qr6cwch4kdcqy-aws-c-common-0.4.59 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A aws-c-common https://github.com/r-ryantm/nixpkgs/archive/010fb6badffd6aaffa04fa3df78d05b9f84f796d.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/dgjd0jhk31fvifsn584qr6cwch4kdcqy-aws-c-common-0.4.59
ls -la /nix/store/dgjd0jhk31fvifsn584qr6cwch4kdcqy-aws-c-common-0.4.59/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
nixpkgs-review encountered an error. Please check the logs at https://r.ryantm.com/log/ for more information.

Maintainer pings
cc @orivej @edolstra for testing.

@GrahamcOfBorg eval

@GrahamcOfBorg eval

@ofborg eval

@ofborg eval

Closing in favor of #107623

Closing in favor of #107623


Alternate functions assignable to unused pins

Rob.
when using a single encoder and button Is it possible to reuse the unused pins as outputs. Can this be configured or available as an option see the attached for idea ( only the relevant section)
thanks
steve
PCF8574.docx

Hi Steve,
Yes using the free pins is possible, there are a number of things that needs some thinking before it can be in the library.
First the user is now free to select which set of the pins is connected to the rotaryEncoder when he chooses to connect only one.
That means there are 4 possible configurations to define the six free pins.
In your case you want to define 3 as input and 3 as output if I understand correctly.
maybe the easiest way to solve this is to use the PCF8574 library side by side with the rotaryDecoder class.
The PCF8574 class you use to control the pins, input, output, HIGH, LOW etc.
You might need to mask pins.
Alternative would be to add two functions to the class to be able to use unused pins.

void write1(pin, HIGH, LOW)
**uint8_t read1(pin);

As I am rather busy I will come back to this issue later next week.

Note to myself:  rotaryDecoderSwitch library should have similar additions.

Thanks Rob. I have hardware to do first so wont get to firmware until next month. In the specific example i have it is as follows  :-B0,B1  quadrature with pull upB2        third input with pull up and multiplexed LedB3,B4,B5 Leds output only but could have switchesB6        multiplex direction output onlyB7        buzzer output onlyThis leaves the quadrature with exclusive pins the next 4 are potentially bidirectional and run with a shadow register for both output and input. B6 is the selection for direction and bit 7 is static output Thanks SteveSent from Outlook for Android

@Steve
Just saw your message. I already created a develop branch as another task took less time.
That allowed me to prototype this one.
The read1(pin) function should work without a problem as it doesn't alter anything.
The write1() function writes a bitmask back and that could cause problems and that need to be tested carefully.
It might interfere with the RE pins e.g. keeping pins LOW or HIGH or ....
I have no hardware setup to test it myself nearby so I just leave it as a develop branch.
Furthermore I did not write an example as the usage is straightforward and an example will probably not match the users configuration. So that is left as an exercise :)

Send me an email with your address and i will put a piece of hardware in the post... @.*** i have a first gen sample that will function SteveSent from Outlook for Android

My email is in the library.properties file


Stop brokers in which connector is interested

Hi,
I found that every time I stop a broker which is the leader of a partition which is streaming to a client build with this library, number of gouroutines  is increasing in the process. This patch makes the connector stop unnecessary broker links when it refreshes metadata.

Merged, thanks!

Thanks for merging. The correct title was "Stop brokers in which connector is not interested"


Expanding commands to not be converted.

Adding more commands with empty args (--) to the COMMANDS_TO_NOT_BE_CONVERTED variable.
Close #1439

Why not change the converter to use kwargs? It's more verbose, but I think it makes the scripts read better.

@akaszynski
Failing style check because of:
src/ansys/mapdl/core/convert.py:43: LARG ==> LARGE
src/ansys/mapdl/core/convert.py:60: SMAL ==> SMALL

There is no option in codespell to ignore a single line, and I don't think it is a good idea to add "LARG" or "SMAL" to exceptions nor the whole file. Therefore, I will merge without style check completed once you approve the PR.

Why not change the converter to use kwargs? It's more verbose, but I think it makes the scripts way more readable. Then we won't run into this issue.

Are you thinking in something in particular? Maybe something like;
converted_script = convert_apdl_block(avoid_commands=["LATT"])

??


superfluos slash

https://github.com/kubernetes/charts/blob/ec99fad17ec5f0764b6712237d3e6b9474581b52/incubator/kafka/templates/statefulset.yaml#L49
hey guys,
nice work!
I was able to partially run this with minikube. I'm kind of a newb to kafka/helm and had to mess around with the statefull sets a little until it was working.
One thing was removing this slash behind the zookeeper dns....
no idea whether it is actually relevant 🗡
hth Martin

The chart has changed a lot in the meantime. Feel free to create another issue if you still have problems.


Enhancing the filtering capabilities on custom Datasources

Hi there !
At present, datasources utilize getModel to generate the required items, fetching all records of that model.
However, there are scenarios where filtering records based on conditions is necessary, rather than displaying all records to the user.
This implies that developers should have control over the query and be able to customize it.
For instance, displaying records related to the logged-in user
Do you believe this is achievable?

fixed #277
thanks for the PR :)


Controller, Router, Route Architecture change

Architecture change
This proposal concerns the controller, router resources. I will also be defining two new non existing resources.
Current architecture
The following section describes my understanding of the intended architecture. I will not be using a "I believe" structure when stating observations of the architecture, I will just be using a declerative style.
Atlas is a cli tool that generates a Typescript server that follows a version of the MVC architecture. Atlas encourages a functional programming style.
Model
The model is defined within the database folder. The database folder defines the different components of the model. The models folder defines MongoDb schemas and models that define how documents are stored in the database. Associated interface types are defined in the interfaces folder. The interactions defines modules around the MongoDb models that will used by the controller.
Controller
The controller is defined in the controllers folder. This is where all business logic is located. In similar express applications, this folder could also be called services. This layer is not coupled to a specific database. It has an aggregation association to the interactions package in the model and a dependency to the interfaces defined in the model.
View
The view is defined in the routes folder. It uses functions in the controller to defined and handles errors in a controlled way (i.e. maps them to a certain http code).
Suggestions
New folder structure

tests/

routes/

src/

routes/

dummy_namespace/

router.ts
dummy_routes.ts
nested_namespace/

router.ts
dummy_routes.ts

controllers/

dummy_controller.ts

database/

models/

dummy_model.ts

interactions/

dummy_interaction.ts

interfaces/

dummy_model_interface.ts

middleware/

dummy_middleware.ts

app.ts
server.ts
.gitignore
README.md
package.json
tsconfig.json
tslint.json

Adding interfaces folder to database
The idea here is to completely decouple your databse from your controllers. Instead of your controllers having direct access to the Document type, the user defines types which exposes the attributes that the controller is allowed to use. Given that you generate the boilerplate, adding this type of interface would not come at that big of a time cost to the cli user.
As a result of this, the atlas generate model command would now need to generate the MongoDb model, the associated interactions module and the associated interface.
Modifying the router
... tired of writing at this point, will reformulate later. The basic idea is...
Router creation
Allow the creationg of namespaces in your routes. Examples

atlas generate router new Would create a new file named new.ts and would connect it the the namespace's router (in this case the top level router).
atlas generate router namespace/new Would create a new file named new.ts and would connect it to the namepsace's router.

Routes creation
Allow route creation with a lot of flags.

atlas generate route new/route --verb "get" --errors customError=411[,...] [...] Would create a new route on the the new router. It would also take care of connecting the route to the server (and generate the swagger yaml).

Notes
This need more thought, I just wanted to give you an elevator pitch.
Notes

In a lot of other projects, the controller folder has the role that the routes folder has in an atlas project.

A few concerns:

Why move interfaces to database? Interfaces aren't used only within the context of databases e.g. users can get data from different sources and would need to type them through interfaces.
Why are app and server outside of the src directory? src is where all the source code for the project should be.
Why do we want to generate namespaces for routers?


Add splitOnRegex

Create a regex version of splitOn.

@cust0dian pointed out that String#split has more modes than Vim. Optional capturing groups are problematic:
> 'foo;;bar;baz'.split(/;(;)?/)
['foo', ';', 'bar', undefined, 'baz']

I see two solutions:

splitOnRegex :: RegExp -> String -> Array String
This option would ignore capturing groups in the pattern:
> splitOnRegex(/;(;)?/, 'foo;;bar;baz')
['foo', 'bar', 'baz']

splitOnRegex :: RegExp -> String -> { gaps :: Array String, matches :: Array (Array (Maybe String)) }
This option would include an array of arrays of captured substrings in the result:
> splitOnRegex(/;(;)?/, 'foo;;bar;baz')
{gaps: ['foo', 'bar', 'baz'], matches: [[Just(';')], [Nothing]]}

I favour the first option, as dealing with Array (Array (Maybe String)) would be annoying.
Regardless of which option we choose we won't be able to use String#split in the implementation.
/cc @jordrake

I would definitely expect  splitOnRegex :: RegExp -> String -> Array String (and Haskell folks seem to agree).
If this is left as good first contribution and @jordrake isn't up for the challenge, I think we should provide example implementation here since I'm not sure there's a trivial, straightforward one.

I have a working implementation:
function splitOnRegex(pattern, s) {
  //  Create a RegExp object with the `global` flag set, so that
  //  RegExp#exec will advance the `lastIndex` property.
  var $pattern = regex(
    'g' + (pattern.ignoreCase ? 'i' : '') + (pattern.multiline ? 'm' : ''),
    pattern.source
  );
  var result = [];
  var lastIndex = 0;
  var match;
  while ((match = $pattern.exec(s)) != null) {
    //  Handle patterns such as /(?:)/ which match the empty string.
    if ($pattern.lastIndex === lastIndex && match[0] === '') {
      if ($pattern.lastIndex === s.length) return result;
      $pattern.lastIndex += 1;
    } else {
      result.push(s.slice(lastIndex, match.index));
      lastIndex = match.index + match[0].length;
    }
  }
  result.push(s.slice(lastIndex));
  return result;
}

Arriving at this version took many iterations. Use a stateless (i.e. non-global) RegExp object seemed wise initially, but 'xxx' would then be split multiple times on /^x/. We must provide the whole string each time and rely on the lastIndex property of a global RegExp object. We even need to manually advance the lastIndex property when it gets stuck on a pattern such as /(?:)/. It's a complex implementation; we'll need both unit and property-based tests to have confidence in its handling of edge cases.

Happy to change my PR to reflect this. Not sure I could do the unit testing (especially the property-based testing) justice on my first pass but that's what comments are for.
If somebody needs this in sooner, I am equally happy to drop my PR and have somebody more experienced take over.

There's no rush, @jordrake. Let's continue to iterate on your pull request. :)


python310Packages.pyswitchbee: 1.6.2 -> 1.7.3

Changelog: https://github.com/jafar-atili/pySwitchbee/releases/tag/1.7.3
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 206075 run on x86_64-linux 1

1 package failed to build:

home-assistant-component-tests.switchbee

2 packages built:

python310Packages.pyswitchbee
python39Packages.pyswitchbee


[Roadmap]: User Settings

added UI site wide user settings in the UI. This feature needs design.
Items to be in user settings:

Set theme Betting | Trading
Set gas price
Set DAI approval amount

Adding this to the roadmap and we'll see where it falls via releases

We're axing this as a feature, removing from roadmap


env property fails in docker environment

Nuxt can't seem to pull environment variables from within a docker environment. When following this example, the baseUrl defaults to the empty string while running in a node docker environment (node:latest docker image), whereas it populated with said env variable if running on a non-containerized host.

AFAIK there is an error in docs. Maybe it's you case, @squarenomad ?
You can see:

Via context.baseUrl
but should be:
Via context.env.baseUrl

I think the docs have error because I was trying to use env variables in nuxt.config in my project and they were stored inside context.env

Hi, We need more details and code snippets to help you.

Hey @squarenomad could you solve that issue? I have the exact same issue!

We are heavily using docker containers for nuxt project at fandogh. There are some facts about different envs:
System wide environment variables (using docker env)
Assessable in SSR side only with process.env but are not defined in client side. (That's obvious why because they should be hardcoded in web pack js bundle working on all clients)
Environment variables passed via nuxt.env[]
Accessable from all webpack-compiles js files using process.env within both client side and server side. In server side they may be overritten using global env variables.
Passing system environment variables to nuxt.env[]
Something like { env: { baseURL: process.env.baseURL } } . This is probably what you want, as it pulls env variables from system environment (CI or Docker) and also adds them to webpack bundle in both client side and server side.
What i suggest
Don't do it at all! As mentioned above the only possible way is using webpack's define env feature but it breaks Twelve Factor principles and at least our standard won't allow this. The reason is because we are hardcodding configs (environment) in production builds. So baseURL can be only and only defined during build time (local or CI) images cannot change it during runtime phase with fancy docker environment variables.
So how to deal with baseURL thing?
Use official axios module with either using relativeURLs or proxy passing to remove API. (proxy's options are server-side so can be customized with docker env too). Or at least try similar approach if really want to make standard docker images :)

@pi0 Do we still have the problem with passing system environment variables during run time ? I have to pass variables during runtime phase and client is not able to pull in the variables. Is there a work around for this issue ?

@thecodr Unfortunately no, that's not a problem but more a fact about client side bundling! For runtime settings, you can easily use vuex store and fill it on server side.

@pi0 Is there any reference for filling vuex store on server side ? I tried several times and failed. Some kind of reference will be really helpful. For us, the entire config resides on s3 in json format. We should load that configuration dynamically into vuex store on server side. We should be able to fill the store even before the site loads completely. Thats exactly the place where I am stuck. The vuex store is being populated after the site is loaded :(

@pi0 This bit me too hard 😅, well we now just build the image using build args and environment settings at build time. Sure it produces host-dependent images and its not pretty but if it works then Its fine at the moment.
In the future, can nuxt set the variables at startup way before the app is starting to serve requests? because like docker, any build produced is host dependent and isn't portable, and hard coding the variables seems to break cohesion between server and client, specially if using the express template settings.

it's definitely an design issue when you replace environment variables at build time and make it impossible to access them at runtime (because they was replaced with plane strings).
We for example need env variables to set microservice configs at runtime inside docker containers. It's not possible to rely on buildtime cars or put that stuff inside the container before shipping it.
Currently we just run the build process inside the deployed container the first time to get the vars. But the that is not a solution as it demands processing time where it dose not belong.
The VueX solution that was mentioned does not feel right ether to be honest.
My question: Could there be a small script that injects That variables without running a full build?

@appinteractive I really hope we finally find some workaround like that script you mentioned. But that injection corrupts chunk hashes! Another idea would be including a generated script in assets/env.js and always add it to <head> this was env.js can inject thing on runtime.
For the microservices problem you mentioned above, i strongly suggest using proxy module. We are using Docker deployments a lot at Fandogh and AUT with this module and it almost solves our problems :)

@pi0 It'll be great if there is an example using proxy module to do the trick.  I think this is a quite common scenario when using nuxt in production. We need to deploy to multiple different QA environment before production, this will really save a lot efforts.
Using the same nuxt app docker image --> passing different --env when running the container  --> get separate backend services depends on the baseURL provided.
Thanks !

Yeah would be interested in an example too. I don’t know how to switch it ether without the correct environment variables. And the proxy only works for url not for the issue of configurations ether.
So there is no way around of correctly implementing environment variables on the server site or at least giving a workaround for that.
Also the proxy solution has issues with high traffic, there was an unresolved issue that is flying around for some years now. (Didn’t recheck the status)

I wrote a blog post about this.
https://vanja.gavric.org/blog/configure-nuxt-js-to-consume-env-variables-during-run-time/

[ Nuxt 1.4.1 ]
I have a case have some similar run-time env problem.
something like I want deploy builded nuxt docker to few server .
I already set some machine and target env in docker run -e command
and I want env apply to vue mounted process , but , I got undefined or env value that when I build nuxt image.
seems like we can't use node env on runtime nuxt.js
@pi0 , sorry about tag you , Can we have some way to set run-time env to nuxt server , and it overwrite webpack env setting on run-time easier ? or it will be nxut 2.0 can implement ?
thanks .

+1 for runtime env vars in Nuxt. This is such a common problem, that should be handled by the framework instead of everyone building their own version of this.

+1 Same problem

+1 same probem in nuxt 2.1

okay. I have strange problem. I am passing env variables from file to docker which passes them down to process.env. Now the problem is the env variables are being passed down in development phase but not in production phase. Isn't it interesting? here is the repo

@Lahori-Jawan env variables are set at BUILD time.

@aldarund no. ARG & ENV both variables are set at build time but ENV varialbes can be used to reset/override values at runtime too e.g. more, but that is not the point here. My container when run in development mode shows different result than when in production mode. you can test that form my repo. In development, I have access to process.env configured in nuxt.config.js but I loose access to it in production. How about images to speak:

@Lahori-Jawan you are missing what im saying. I`m not talking about docker envs. Nuxt bundle environment variables into bundle at build time. So what is matter what variables are set when u do nuxt build, not a nuxt start

@aldarund I surely missed that. I will try it & update you on this.

If u need runtime env variables you can use https://github.com/samtgarson/nuxt-env  ( ⚠ WARNING: As with the config.env option in Nuxt config, environment variables used in nuxt-env are exposed client side, so don't use it for storing secrets! ⚠ (

@aldarund I think the only best option I have right now is to NOT run build while building the image instead do it when creating container. Do you think its good practice? which I guess is good when there is no 2nd way of doing this.

my use case is to set live environment credentials for nuxt-apollo module to set httpEndpoint for which apollo module requires to set via plugins.


Change in RenderingDevice decode api and docs

My goal here is to add some method on RenderingDevice for ease of use in godot script. As it is easy to push a structure in c++ on GPU it's not as easy in godot script so I just made some method to be able to push/retrieve a PackedVector2Array, a PackedColorArray, a PackedFloat32Array... I think it is useful and I am actually using it for compute shaders ;)
RenderingDevice::buffer_update:
It is now two template functions.
One to give an array of element and one to give only one element for ease of use.
Those two template functions are exposed as specialized functions in gdscript for the different type of array and their type T.
PackedByteArray::to_XXX_array:
Became a template function to avoid code duplication and then the content of this function have been moved to a Vector ctor to make it more widely available.
Add three more specialization of this method for vector2, vector3 and color as it is commonly used type in shader.
Discussion started here but nobody seems to have something to say against it : https://github.com/godotengine/godot-proposals/issues/3042
Previous PR & godot-proposals linked to this issues :
https://github.com/godotengine/godot-proposals/issues/2993
https://github.com/godotengine/godot/pull/50535

The main error is:
'framebuffer_get_format' overrides a member function but is not marked 'override' [-Werror,-Winconsistent-missing-override]
With different functions involved.
I suppose that the error has not been found before because the file has not been modified since a while.
Should I correct it even if I didn't altered the involved functions ?

I suppose that the error has not been found before because the file has not been modified since a while.
Should I correct it even if I didn't altered the involved functions ?

Yes, please correct it, but in a separate PR.

Fellow devs, this PR need review ! :)

With some code examples, the gain will be more visible.
With Godot's master:
var vectors : PackedVector2Array
# fill vectors
rd.buffer_update_vector2(buffer_vectors, 0, vectors.size() * 8, vectors.to_byte_array())

var mouse_x : PackedFloat32Array = [ get_global_mouse_position().x ]
rd.buffer_update_float32_element(buffer_mouse, 0, 4, mouse_x.to_byte_array())
# or
var mouse_y : PackedByteArray = []
mouse_y.resize(4)
mouse_y.encode_float(get_global_mouse_position().y)
rd.buffer_update_float32_element(buffer_mouse, 4, 4, mouse_y)

⚠ Reminder : to_byte_array() -> mem_cpy()
With this PR:
var vectors : PackedVector2Array
# fill vectors
rd.buffer_update_vector2(buffer_vectors, 0, vectors.size(), vectors)
rd.buffer_update_float32_element(buffer_mouse, 0, get_global_mouse_position().x)
rd.buffer_update_float32_element(buffer_mouse, 1, get_global_mouse_position().y)

@clayjohn @Chaosus This PR need reviews guys.
I talked with @reduz and @clayjohn on the contributor chat and they are discussing the interest of RenderingDevice::buffer_update_XXX_element methods. They think that it may not be a good idea to encourage to send a buffer element by element (it was not my intention and not the purpose of this method but effectively it may encourage that).

If I have to be entirely and completely honest, I still think that this PR adds more bloat than proper use and feel inclined to not merge it. My feeling is that its easier to just bind a void* function, worst case, to the new binder so if you use from an extension you dont have to go through PackedArray, but using these kind of things from GDScript and expecting optimization seems far fetched.
Let me think more time about it, because I am still unconvinced.

To Clarify I thought initially this PR was just about adding a unique single buffer_update function that took Variant as buffer argument, and then called buffer_update depending on whether any of the types contained was valid as a buffer and error otherwise. This was a simple change, so to me its fine.
This PR now ends up to touching too many things and making the code more complex for something that is not warranted to be an actual need from users.

I suggest this API instead, which does minimum changes to the driver.
//instead of
virtual Vector<uint8_t> buffer_get_data(RID p_buffer) = 0; //this causes stall, only use to retrieve large buffers for saving
saving
//
virtual uint64_t buffer_get_size(RID p_buffer) const = 0;
virtual void buffer_write_data(RID p_buffer,uint64_t p_offset,uint64_t p_size,void *p_ptr) = 0; //not exposed to binder
Vector<uint8_t> buffer_get_data(RID p_buffer); //exposed to binder, but not virtual
Variant buffer_get_data_var(RID p_buffer,Variant::Type); //exposed to binder, but not virtual
//also add
void buffer_update_var(RID p_buffer,uint64_t p_offset, uint64 p_size, const Variant& p_data); //not virtual

@reduz I think you are right, I may have been carried away while coding ^^"
It's a good argument that sending/retrieving data on GPU is indeed slow and one memcpy will not make a huge difference.
I will try to rewrite the PR to have modification only in the base class RenderingDevice.

I will close this for now due to the above discussion. Thanks for contributing :)
Feel free to open a new PR once you reworked it (and add a link to this one in the description), or ping us to open this one.


Revert "feat: add dotnet-type (#8)"

This reverts commit a072fa3cb6d7282d17b1641d9b82429fed753657.
The type dotnet is invalid currently and should not have been added to our fork.
Related to: https://github.com/anchore/syft/issues/1622

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 80.087%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
185

Relevant Lines:
231

💛  - Coveralls


Measurement.Data.data.data.data

The data class data has the field data and name, but the since xarrays support the name attribute as well, I wonder why we have this kind of duplication? It makes code working with Measurements look strange, since we need to access data twice to obtain the actual xarray. Further if I want to access the raw data it gets even worse :)
We could just define a measurement to directly contain a "plain", although pretty high-level, xarray, right?

I am currently rewriting the MeasurementChain related stuff in the context of #63. Removing this "construct" is also on the agenda... even though there is no such thing as too much data 😄

yes it can get confusing quickly, I think we should try to stick to existing naming conventions if possible (so .data should only be an ndarray-like the way it is for xarray for example). We could find a more fitting attribute name for xarray objects as well
We should discuss a rework of the measurment module soon, maybe you can share your Ideas here or in #63 @vhirtham ?

This should basically be solved with PR #326 . However, there are some occasions where we get a data.data, for example in the TimeSeries. This always occurs if a data function returns an xarray type and you want to have the raw data. Not sure if we want to go any further, but I think it should be okay.

maybe rename instances that access xarray objects to something else besides data like dataarray, variable or similar? and try to keep it consistent for different classes

Seems reasonable

please review if still relevant after latest PRs @vhirtham


fixes bug 1132249 - Switch to using AddMediaLite

Coverage remained the same at 90.33% when pulling f0e485bf43aafdf56bd738918442cf9777c39b4f on peterbe:bug-1132249-switch-to-using-addmedialite into 9d06709f5b8a8ee99d0d2d034a74da23b84b376e on mozilla:master.

Coverage remained the same at 90.33% when pulling f0e485bf43aafdf56bd738918442cf9777c39b4f on peterbe:bug-1132249-switch-to-using-addmedialite into 9d06709f5b8a8ee99d0d2d034a74da23b84b376e on mozilla:master.


fix issue that occurs from change in serialization format in Julia 1.9

See discussion in https://github.com/JuliaLang/julia/issues/48078 (and https://github.com/PainterQubits/Unitful.jl/pull/600)

Thanks!


Class is not registered: org.datasyslab.geospark.spatialRddTool.StatCalculator

Expected behavior
Actual behavior
Steps to reproduce the problem
Settings
GeoSpark version =1.3.0
Apache Spark version = 2.4.4
JRE version = 1.8
scala version = 2..2.7
API type = Scala

//    conf.set("spark.kryo.registrationRequired", "true")


Mistype in LSAISO

Mistyped LSASIO instead of LSAISO

@artursg : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


add 'Stateless' mode

[x] add new mode called 'Stateless'
[x] when enabled, local files should be deleted once they are uploaded and confirmed to be stored on Google Cloud Storage (GCS).

To to be clear, we do NOTHING with any media that was added without wp-stateless. We want to avoid having any sort of batch process that may remove older media.

@andypotanin Can you please clarify. If we set the mode to stateless, and rerun the "sync/upload" wizard, will it delete the local files? I have a site I'd like to migrate to stateless mode (to free up space on the server), and I'm trying to work out how to get it to delete the existing media files after upload.
Thanks

@timwhite Stateless mode will delete the local file upon successful upload to Google Cloud Storage. This will only affect newly uploaded files after configuring this Mode setting.
In your situation, you could set mode to Stateless, run the regeneration/sync option, and delete the files manually via SFTP or SSH. Be sure to have a backup too - just in case.


Make QAs application specific

Separate QA definition from the metacontroller, make QA application specific:
the developer should define the QAs needed for the application: energy, safety or qa_x
For this, we need to create the generic QA class in tomasys so that we can have individuals in the application model for the application specific QAs

Done in 5263ad6da4a7507754caf6dbf0068e774c223b65 and
https://github.com/rosin-project/mc_mdl_tomasys/tree/mvp
https://github.com/rosin-project/mc_mdl_abb/tree/mvp


fix webapp vite for aks template

fix for: https://github.com/Azure/azure-dev/issues/4108#issuecomment-2226282647
The todo ask template was missing the hook which sets the api env vars for building the web application to be deployed.

Repoman Generation Results
Repoman pushed changes to remotes for the following projects:
Project: todo-nodejs-mongo-aks
Remote: azure-samples-staging
Branch: pr/4117
You can initialize this project with:
azd init -t Azure-Samples/todo-nodejs-mongo-aks -b pr/4117

View Changes | Compare Changes

using https://github.com/Azure/azure-dev/pull/4069 instead


Fix link to util/populate_roi.py

Fix build failure:
- 404 Client Error: Not Found for url: https://github.com/ome/openmicroscopy/blob/develop/components/tools/OmeroPy/src/omero/util/populate_roi.py

Maybe these repos need adding into omero/conf.py's omero_extlinks.

Conflicting PR. Removed from build OMERO-docs#181. See the console output for more details.
Possible conflicts:

PR #2015 will-moore 'web app docker template'

omero/developers/Web/CreateApp.rst

--conflicts

Conflicting PR. Removed from build OMERO-docs#183. See the console output for more details.
Possible conflicts:

PR #2015 will-moore 'web app docker template'

omero/developers/testing.rst

--conflicts

The OMERO-docs job is green again: https://merge-ci.openmicroscopy.org/jenkins/job/OMERO-docs/184/

Generally looks good. Any thoughts about @mtbc's suggestion in https://github.com/ome/ome-documentation/pull/2020#issuecomment-530393138 to define new web_source and py_source extlinks in https://github.com/ome/ome-documentation/blob/develop/omero/conf.py#L99?

I'm not sure how web_source and py_source extlinks would help us.
I usually find that I need to browse to the source anyway, to check I have the link right. Then it's easier for me to put that in a regular link than to remember or find an example of how to use extlinks. So I don't find that they make docs writing any easier. Do they help us in any other way (back-links etc)?

@will-moore using extlinks is the strategy that had been decided collectively during the OMERO 5.5 work. Understood that it would constitutes an extra simplification step in the workflow you describe above although we are talking of an overhead of a few seconds.
The primary benefit is the management of the component versions since the extlinks point at tags of the source code. As components get released rapidly, this allows us to keep the entire documentation in sync by updating a single file rather than reviewing all pages - see https://github.com/ome/ome-documentation/pull/2003 for an example.


Should it be Signal.Derived rather than Signal.Computed?

This is name bike-shedding but I thought it would be worth bringing up.
I ask this as technically effects are computed signals too. It made sense to use Signal.Computed when we coupled both concepts onto the same class, but given we're breaking them out and they have different heuristics around what can be written and read from, it also makes sense to tailor there naming IMO. Pure derived state has its foundations in many reactive papers and prior work so I feel it lends itself nicely here.

I'm one of those weird people who spend way too much time on etymology.
I think it's important that the word describes the semantics it represents precisely.

derive | dɪˈrʌɪv | obtain something from (a specified source)
From late Middle English (in the sense ‘draw a fluid through or into a channel’): from Old French deriver or Latin derivare, from de- ‘down, away’ + rivus ‘brook, stream’.

Literally "downstream", or "obtain from downstream". Note the allusion to fluids (dataflow).

compute | kəmˈpjuːt | reckon or calculate (a figure or amount)
From French computer or Latin computare, from com- ‘together’ + putare ‘to settle (an account)’.

Much more general. Happens all the time in code.

memo | ˈmɛməʊ | a written message within a business or organization
Abbreviation of memorandum: from Latin, literally ‘something to be brought to mind’, gerundive of memorare. The original use was as an adjective, placed at the head of a note of a record made for future reference.

Or in this case, memoization, a technique for optimizing functions. Also more general.
So I do prefer derive, although it is used more generally when talking about functions, as NullVoxPopuli said.

Let the debate recommence!

On the topic of naming things, is Computed not often considered State? I find it a bit clearer to name it for what it is as opposed to how it's used..
For example the Streams API has ReadableStream and WriteableStream. I think Computed (or Derived) similarly makes sense, it's describing what it is (I don't have a strong opinion between those)
But Signal.State doesn't quite feel right to me. I could easily see myself wanting to do this often:
const state = new Computed(() => ...);

How about one of these alternatives?

Signal.Init (i.e. a signal with an initial value)
`Signal.Mutable (i.e. a signal that can be mutated)
`Signal.ReadWrite (same idea, but following streams API conventions)

Static?
This proposal is a tough one, it involves both cache invalidation and naming things.


feat!: add MultiSend deactivation

Description

closes: #868
Activate this feature on bankpluskeeper directly instead of using config. Because this feature is not supposed to change by an outside party. It need to use intentionally when build an App instance.
Motivation and context

How has this been tested?

Screenshots (if appropriate):
Checklist:

[x] I followed the contributing guidelines and code of conduct.
[x] I have added a relevant changelog to CHANGELOG.md
[x] I have added tests to cover my changes.
[ ] I have updated the documentation accordingly.
[ ] I have updated API documentation client/docs/swagger-ui/swagger.yaml

@tkxkd0159 , unittest also should change.
https://github.com/line/lbm-sdk/blob/a7557b1d1029fcf9c112f3dc0cf94d158e197bc7/x/bankplus/keeper/keeper_test.go#L297-L299


[Bug] improve performance of pdf2parquet

Search before asking

[X] I searched the issues and found no similar issues.

Component
Other
What happened + What you expected to happen
Extracting text from PDF into parquet seems slow.  It is processing 1 page / second.  So if a PDF has 300 pages, it takes 300 seconds (5 mins)
This negatively affects the user experience, as PDF2PQ is usually one of first few steps in many workflows.
Reproduction script
data : https://github.com/sujee/data-prep-kit/tree/perf-1-pdf2pq/test/perf-pdf2pq/input
(These PDFs are about 100 pages each)
Instructions and minimal code to reproduce the problem are here : https://github.com/sujee/data-prep-kit/tree/perf-1-pdf2pq/test/perf-pdf2pq
instructions (README.md) : https://github.com/sujee/data-prep-kit/blob/perf-1-pdf2pq/test/perf-pdf2pq/README.md
A py-spy generated speedscope file is attached.  It can be viewed at https://www.speedscope.app/
test_pdf2pq_py.speed.txt
Anything else
No response
OS
Ubuntu
Python
3.11.x
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

@dolfim-ibm Can you suggest any updates on this pls?

Soon we will update DPK to use the new Docling v2. As part of the new feature (together with support for docx, html, pptx, etc) we have a new parse which is about 10x faster. See https://github.com/DS4SD/docling-parse/?tab=readme-ov-file#performance-benchmarks. Note, this is not the speed up of the full pipeline, but one of the important pieces.
Medium term, we are actually running heavy benchmarks to identify the characteristic timing and compare with other tools.

Thank you @dolfim-ibm When do you expect to integrate this change?

Should be doable this week.

@dolfim-ibm is Docling v2 supported on windows natively?

@dolfim-ibm is Docling v2 supported on windows natively?

yes, this is supported since v1.17.0.

The version installed with https://github.com/IBM/data-prep-kit/pull/756 should now be faster (20-30%).
Additionally, you could also use the parameter bitmap_area_threshold to run OCR only of large images.

this is the fraction of the bitmap area to the page area. if the ratio is larger than the threshold, the image will be processed with OCR, otherwise skipped.
the default value is 0.05=5%
the default should already get rid of small logos, etc but you can try a value of 0.5=50% for skipping other embedded bitmap images which are not needed


RocksDB Produces fatal error on shutdown

Description
When shutting down teku, the following error occurred:
Teku is shutting down
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x0000000136e6c988, pid=6007, tid=46595
#
# JRE version: Java(TM) SE Runtime Environment (11.0.3+12) (build 11.0.3+12-LTS)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (11.0.3+12-LTS, mixed mode, tiered, compressed oops, g1 gc, bsd-amd64)
# Problematic frame:
# C  [librocksdbjni11922200677829895721.jnilib+0x5d988]  rocksdb::GetColumnFamilyID(rocksdb::ColumnFamilyHandle*)+0x8
#
# No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/mbaxter/code/consensys/teku/build/hs_err_pid6007.log
[thread 115975 also had an error]
[thread 64003 also had an error]
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

Error report: hs_err_pid6007.log
Teku restarted fine after this error.

one easy way to reproduce:

start syncing onyx, on v4 db with 2048 slot storage.
once its got more than a couple of hundred slots, do a fetch of /beacon/state at around 300.
you'll have several seconds to then CTRL+C the teku process and a SEGV will happen.

Running into a similar error again on shutdown.  I think we need a similar fix in methods that modify rocksdb Transaction's.
Error log: hs_err_pid32789.log


[misc] improve typings for isBetween methods

Based on mentioned supported methods in docs example here:

https://momentjs.com/docs/#/query/is-between/ .  isBettween can contains granularity value null. But this is not mentioned in d.ts file so after adding null as possible value below example will work in typescript as it is.
moment('2016-10-30').isBetween('2016-10-30', '2016-12-30', null, '()'); //false

The typings tests fail here.

Thanks, @marwahaha I will take a look and fix this.


feat(api-proxy): [WEBS-000] handle getFlags function with post methods to support edge-proxy

JIRA Ticket Link: https://pomelofashion.atlassian.net/browse/RICK-497

What problem does this PR solve?

The original code from flagsmith doesn't support api with edge-proxy

How does this PR solve it?

Fix the code to support edge-proxy api
Release v2.0.0

TERMS & CONDITIONS: By pressing submit, I do hereby declare that my PR meets the following requirements of all code on Pomelo.:

The Definition of Done (However, if this PR is for a partial story, certain pieces of that may be deferred until the story is complete. Please complete all pieces related directly to this one PR.)
The Code-Review Guidelines
The UI Styleguide and Code Conventions

:tada: This PR is included in version 1.1.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: