Feed stops after some short time (frames)

After fixing the setup by issuing the command modprobe bcm2835-v4l2 in a terminal, I was able to get a camera feed, but only for a short time.
After that, the browser screen goes blank, and I eventually get a sad smiley with some 'sorry' error message.
I tried changing a line to await asyncio.sleep(5)
from I think 0.2? to slow things down, thinking it might be too much data for the pi zero to handle.
It didn't actually slow the framerate down from what I could tell...
Some logs follow.
16.12.19 22:59:17 (-0800) <balena-cam>
16.12.19 22:59:17 (-0800) <balena-cam> #############################################################
16.12.19 22:59:17 (-0800) <balena-cam> Authorization is disabled.
16.12.19 22:59:17 (-0800) <balena-cam> Anyone can access your balenaCam, using the device's URL!
16.12.19 22:59:17 (-0800) <balena-cam> Set the username and password environment variables
16.12.19 22:59:17 (-0800) <balena-cam> to enable authorization.
16.12.19 22:59:17 (-0800) <balena-cam> For more info visit:
16.12.19 22:59:17 (-0800) <balena-cam> https://github.com/balena-io-playground/balena-cam
16.12.19 22:59:17 (-0800) <balena-cam> #############################################################
16.12.19 22:59:17 (-0800) <balena-cam>
16.12.19 22:59:17 (-0800) <balena-cam> ======== Running on http://<IP_ADDRESS>:80 ========
16.12.19 22:59:17 (-0800) <balena-cam> (Press CTRL+C to quit)
16.12.19 23:01:09 (-0800) <balena-cam> Exception in callback Transaction.__retry()
16.12.19 23:01:09 (-0800) <balena-cam> handle: <TimerHandle when=326.478631283 Transaction.__retry()>
16.12.19 23:01:09 (-0800) <balena-cam> Traceback (most recent call last):
16.12.19 23:01:09 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
16.12.19 23:01:09 (-0800) <balena-cam>     self._callback(*self._args)
16.12.19 23:01:09 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
16.12.19 23:01:09 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
16.12.19 23:01:09 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
16.12.19 23:01:09 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
16.12.19 23:01:09 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished result=(Message(messa...\tN\xcfF\x10'), ('<IP_ADDRESS>', 58055))>```

16.12.19 22:23:45 (-0800) <balena-cam> Exception in callback _SelectorTransport._call_connection_lost(None)
16.12.19 22:23:45 (-0800) <balena-cam> handle: <Handle _SelectorTransport._call_connection_lost(None)>
16.12.19 22:23:45 (-0800) <balena-cam> Traceback (most recent call last):
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
16.12.19 22:23:45 (-0800) <balena-cam>     self._callback(*self._args)
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/selector_events.py", line 662, in _call_connection_lost
16.12.19 22:23:45 (-0800) <balena-cam>     self._protocol.connection_lost(exc)
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/ice.py", line 161, in connection_lost
16.12.19 22:23:45 (-0800) <balena-cam>     self.receiver.data_received(None, self.local_candidate.component)
16.12.19 22:23:45 (-0800) <balena-cam> AttributeError: 'NoneType' object has no attribute 'component'
16.12.19 22:23:45 (-0800) <balena-cam> Exception in callback _SelectorTransport._call_connection_lost(None)
16.12.19 22:23:45 (-0800) <balena-cam> handle: <Handle _SelectorTransport._call_connection_lost(None)>
16.12.19 22:23:45 (-0800) <balena-cam> Traceback (most recent call last):
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
16.12.19 22:23:45 (-0800) <balena-cam>     self._callback(*self._args)
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/selector_events.py", line 662, in _call_connection_lost
16.12.19 22:23:45 (-0800) <balena-cam>     self._protocol.connection_lost(exc)
16.12.19 22:23:45 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/ice.py", line 161, in connection_lost
16.12.19 22:23:45 (-0800) <balena-cam>     self.receiver.data_received(None, self.local_candidate.component)
16.12.19 22:23:45 (-0800) <balena-cam> AttributeError: 'NoneType' object has no attribute 'component'

Output from top seems like kswapd0 takes over CPU usage right when the thing fails. I've tried changing the GPU memory settings, let's see how that goes.

Yep, by far the longest run I've had yet (and it's still running as I post this comment) has come by overriding device configuration BALENA_HOST_CONFIG_gpu_mem_512 value from 256 to 128 (from here: https://www.balena.io/docs/learn/develop/hardware/i2c-and-spi/#raspberry-pi-camera-module )

Some noise in the logs and a broken feed that seemed to work on refresh:
17.12.19 01:55:47 (-0800) <balena-cam> handle: <TimerHandle when=1585.787987345 Transaction.__retry()>
17.12.19 01:55:47 (-0800) <balena-cam> Traceback (most recent call last):
17.12.19 01:55:47 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
17.12.19 01:55:47 (-0800) <balena-cam>     self._callback(*self._args)
17.12.19 01:55:47 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
17.12.19 01:55:47 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
17.12.19 01:55:47 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
17.12.19 01:55:47 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
17.12.19 01:55:47 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished result=(Message(messa...\x97"ZM\x1f]'), ('<IP_ADDRESS>', 58452))>
17.12.19 01:57:15 (-0800) <balena-cam> Exception in callback Transaction.__retry()
17.12.19 01:57:15 (-0800) <balena-cam> handle: <TimerHandle when=1675.406446503 Transaction.__retry()>
17.12.19 01:57:15 (-0800) <balena-cam> Traceback (most recent call last):
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
17.12.19 01:57:15 (-0800) <balena-cam>     self._callback(*self._args)
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
17.12.19 01:57:15 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
17.12.19 01:57:15 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
17.12.19 01:57:15 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished exception=TransactionFailed(Message(message_method=Method.BINDING, message_class=Class.ERROR, transaction_id=b'\x9a\xe6\xa6\xc3\xfd\xf3\xd8j\x86\xd7OE'),)>
17.12.19 01:57:15 (-0800) <balena-cam> Exception in callback Transaction.__retry()
17.12.19 01:57:15 (-0800) <balena-cam> handle: <TimerHandle when=1676.390663887 Transaction.__retry()>
17.12.19 01:57:15 (-0800) <balena-cam> Traceback (most recent call last):
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
17.12.19 01:57:15 (-0800) <balena-cam>     self._callback(*self._args)
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
17.12.19 01:57:15 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
17.12.19 01:57:15 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
17.12.19 01:57:15 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished exception=TransactionFailed(Message(message_method=Method.BINDING, message_class=Class.ERROR, transaction_id=b'?\xf1`\x96;\xcb*\xb0(\xa4kG'),)>
17.12.19 01:57:15 (-0800) <balena-cam> Exception in callback Transaction.__retry()
17.12.19 01:57:15 (-0800) <balena-cam> handle: <TimerHandle when=1676.409545741 Transaction.__retry()>
17.12.19 01:57:15 (-0800) <balena-cam> Traceback (most recent call last):
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
17.12.19 01:57:15 (-0800) <balena-cam>     self._callback(*self._args)
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
17.12.19 01:57:15 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
17.12.19 01:57:15 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
17.12.19 01:57:15 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
17.12.19 01:57:15 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished exception=TransactionFailed(Message(message_method=Method.BINDING, message_class=Class.ERROR, transaction_id=b'\xc5\xec\xb5x\xe05\xe4\x8d\xde~\xc7r'),)>
17.12.19 02:00:38 (-0800) <balena-cam> Exception in callback Transaction.__retry()
17.12.19 02:00:38 (-0800) <balena-cam> handle: <TimerHandle when=1874.514632089 Transaction.__retry()>
17.12.19 02:00:38 (-0800) <balena-cam> Traceback (most recent call last):
17.12.19 02:00:38 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/events.py", line 126, in _run
17.12.19 02:00:38 (-0800) <balena-cam>     self._callback(*self._args)
17.12.19 02:00:38 (-0800) <balena-cam>   File "/usr/local/lib/python3.5/dist-packages/aioice/stun.py", line 272, in __retry
17.12.19 02:00:38 (-0800) <balena-cam>     self.__future.set_exception(exceptions.TransactionTimeout())
17.12.19 02:00:38 (-0800) <balena-cam>   File "/usr/lib/python3.5/asyncio/futures.py", line 360, in set_exception
17.12.19 02:00:38 (-0800) <balena-cam>     raise InvalidStateError('{}: {!r}'.format(self._state, self))
17.12.19 02:00:40 (-0800) <balena-cam> asyncio.futures.InvalidStateError: FINISHED: <Future finished result=(Message(messa...d1m\xb8\x12>'), ('<IP_ADDRESS>', 62341))>```

Also, this is after uncommenting the line commented here
```if __name__ == '__main__':
    #checkDeviceReadiness() ```

@robot-army since you don't have this problem anymore do you mind closing this? Also I'd recommend always using the latest available version of balenaOS.

I think this needs something for the README to fix the GPU memory sizes, but I'll try again with a new version if I get a moment and see if that fixes it. Thanks for your help!

@robot-army I am going to close this, if you still have the issue feel free to re-open it.


soc: silabs: Update HAL for Series 2

Update HAL for Series 2 devices to Simplicity SDK 2024.6.
This HAL deprecates sl_device_init_emu(). Equivalent functions are now performed by sl_power_manager_init().
This HAL adds a memory manager that requires initialization at boot using sl_memory_init().
For now, soc.c remains compatible with both the Gecko SDK based HAL for Series 0 and 1, and the Simplicity SDK based HAL for Series 2. In a future commit, soc.c will be split between the two platforms.

@asmellby this seems to regress on my xg27 board. With current main I've able connect to it with the peripheral_hr app and browse the GATT services. With this PR I've able to discover the device, but connecting seems to hang forever. The exact build command I'm using:
west build -b xg27_dk2602a/efr32bg27c140f768im40 samples/bluetooth/peripheral_hr

this seems to regress on my xg27 board

One thing worth noting: what I physically have is a main board + radio board combo, however the build target is a dev kit - could that make a difference?

Just tried with an xg24 radio board (for which there does exist an appropriate build target), and the result is the same. Output from the console looks like this:
*** Booting Zephyr OS build v3.7.0-3261-g0c62a2fa29c7 ***
[00:00:00.012,000] <wrn> bt_hci_core: Controller to host flow control not supported
[00:00:00.013,000] <wrn> bt_ecc: ECC HCI commands not available
[00:00:00.013,000] <inf> bt_hci_core: Identity: 80:4B:50:79:59:03 (public)
[00:00:00.013,000] <inf> bt_hci_core: HCI: version 5.4 (0x0d) revision 0x0000, manufacturer 0x02ff
[00:00:00.013,000] <inf> bt_hci_core: LMP: version 5.4 (0x0d) subver 0x0000
Bluetooth initialized
Starting Legacy Advertising (connectable and scannable)
Advertising successfully started
Checking LED device...done.
Configuring GPIO pin...done.
Start blinking LED...
Connected
Stop blinking LED.
Disconnected, reason 0x08
Starting Legacy Advertising (connectable and scannable)
Start blinking LED...
Connected
Stop blinking LED.
Disconnected, reason 0x08
Starting Legacy Advertising (connectable and scannable)
Start blinking LED...
Connected


Maths quiz app with number of questions choice

Describe the project you want to add with tech stack
Maths quiz app with number of questions choice
Screenshots (optional)

@MohitGupta121


PaginatorInterface and paging data

What steps will reproduce the problem?
Currently, paginator classes have no common part, so they cannot be handled uniformly. In addition, implementing a custom paginator is difficult if it comes in this form eg. in the view layer.
I think the first common part should be done with the data needed for paging and the restore from data.
What is the expected result?
$paginatorData = $paginatior->toParameters();
...
$paginator = $pagainator->withParameters($paginatorData);

What do you get instead?
Generate data for paging links:
$pages = [];
if($paginator instanceof KeysetPaginator) {
  $data = [
    'lastValue' => $paginator->getLast(),
    ...
  ];
   ...
} elseif($paginator instanceof OffsetPaginator) {
  ...
} else {
  // custom paginator???
}

...and restoring data to the paginator.
Suggest
interface Paginator {
  function toParamters(int $relativePage): ?array;
  function withParameters(aray $data);
}

example
class KeysetPaginator implements Paginator {
  public function toParameters(int $relativePage): ?array {
    ...
  }
  public function withParameters(array $data): self {
    $new = clone $this;
    if(isset($data['lastValue'])) {
      $new = $new->withLast($data['lastValue']);
    }
    ...
    return $new;
  }
}

$relativePage in toParamters($relativePage)
I suggest that the data be retrieved with a relative number of pages so that the paging view can retrieve the data needed for a page from any distance. If the paginator does not support something, it returns NULL.
example:
$pageData = [];
$relativePages = array_merge([
  PHP_INT_MIN /* first page */,
  PHP_INT_MAX /* last page */,
],
  range(-5, 5)
  // N-5, N-4, N-3, N-2, N-1,
  // 0 (current page),
  // N+1, N+2, N+3, N+4, N+5,
);

foreace($relativePages as $relativePage) {
  $data = $paginator->toParameters($relativePage);
  if($data === null) {
    /* not available or not supported */
    continue;
  }
  $pageData[$relativePages] = $data;
}

// forming page data keys and creating eg URL

Additional info
Required for:

page view
TokenPaginator #5
adding custom paginator

Q
A

Version
1.0.0 under development

PHP version
-

Operating system
-

Currently, paginator classes have no common part, so they cannot be handled uniformly.

That is intentional since widgets for offset pagination and keyset pagination are meant to be different.
With offset pagination it's a "classic" 1, 2, 3, 4, 5 ... 324. We are specifying page number and can jump to any page right away. With keyset pagination it's usually previous / next. We need to know first or last item ID in order to go to next or to previous page. Not only links are formed differently, there's slight difference in UI as well.

That is intentional since widgets for offset pagination and keyset pagination are meant to be different.

This is very wrong ideology.
The minimum condition for a paginator is to be able to the next page. This is already stated in DataReaderInterface, since the iterator can only move forward. As with reading almost all technical tools, e.g. cassette tape, database scanner, file readers.
https://github.com/yiisoft/data/blob/3d4a407268d6712d375662b9c409bc854bb2dcbe/src/Reader/DataReaderInterface.php#L15

With offset pagination it's a "classic" 1, 2, 3, 4, 5 ... 324. We are specifying page number and can jump to any page right away. With keyset pagination it's usually previous / next. We need to know first or last item ID in order to go to next or to previous page. Not only links are formed differently, there's slight difference in UI as well.

Error in this ideology through real examples.
Example 1.: ListView + auto scroll down pagination view.
I use a third-party API like OffsetPaginator. It makes perfect sense to use the OffsetPaginator class as there may be an end page where I use "normal"' paging as well. But I can't use it, because the auto scroll down view does support only KeysetPaginator. This is a meaningless restriction because the next and previous pages have the same meaning in both cases, and the current paginator classes know this.
nextPage(KeysetPaginator) == nextPage(OffsetPaginator) The result is a list of the following elements, regardless of the input parameter requirement.
Example 2.: ListView + "traditional" pagination view
I want to scroll through a large dataset, e.g. database but displaying a "traditional" pagination view configured as follows:

No first / last page needed
Show only next and previous page links
like https://stackoverflow.com/a/56596710/6856708

Can't I just use the much more optimal KeysetPaginator because it is wired to the link pager view widget at the OffsetPaginator framework level?
In summary:
No matter what kind of paging, relative paging should be supported by all pagination, even if limited.
Suggest
interface Paginator {
  // It supports both relative paging and absolute paging, covering all paginator implementations.
  function toParamters(int $pageNumber, bool $asRelative): ?array;
  // This is always useful, as you must be able to process your own data.
  function withParameters(aray $data);
}

Normally, using relative paging is recommended as it is more widely supported. PHP_INT_MIN and PHP_INT_MAX can be used for the first and last pages, and zero for the current page.
A dropdown selector page, on the other hand, only supports pagers like OffsetPaginator, because I can specify the current page number from the drop-down list, which requires absolute paging.

No matter what kind of paging, relative paging should be supported by all pagination, even if limited.

OK, that's a good goal. Makes sense.
The interface suggested won't work though because for keyset paginator we have no idea on how to get parameters from page number. We can, of course, get all pages one by one starting from the first page but that'd defeat the purpose of such paginator. I'd suggest the following interfaces:
interface Paginator
{
    public function read(): iterable;
    public function getNextPageParameter(): ?string;
    public function getPreviousPageParameter(): ?string;
    public function withNextPageParameter(string $parameter): self;
    public function withPreviousPageParameter(string $parameter): self;
}

This interface is limited.

The interface suggested won't work though because for keyset paginator we have no idea on how to get parameters from page number. We can, of course, get all pages one by one starting from the first page but that'd defeat the purpose of such paginator. I'd suggest the following interfaces:

I admit there are shortcomings in my idea, but I still want to make the pagantor interface data completely transparent.
paginator.withParameters(paginator.toParameters(...)) == paginator
This statement should always be true regardless of the complexity of the data used by the paginator. The paginator must handle its parameters by itself. The other layers should not initiate any specific application.
I'm still thinking because the only problem with the current one is that you can't get the absolute page number. But it could be defined in the Offsetable interface.

This interface is limited.

That's the goal to make it both not too vague allowing anything and flexible at the same time.

The paginator must handle its parameters by itself. The other layers should not initiate any specific application.

I don't think that is possible. How would GridView know which parameters are to pass to paginator to get to next page then? In my interface when generating next/previous links it would use getNextPageParameter() and getPreviousPageParameter(). We could extend that to be parameters instead of parameter and use associative arrays but I see no use for it.

The importance of array management is that the paginator can pass on its own data keys and parse them.
$paginator = $paginator->withParameters($_GET);
$data = $paginator->read();
$previousUrl = 'prefix...'.http_build_query($paginator->toParameters(-1, true));
// previous url with 'firstValue' => ...
$nextUrl = 'prefix...'.http_build_query($paginator->toParameters(1, true));
// next url with 'lastValue' => ...

// in next request
$paginator = $paginator->withParameters($_GET);
$data = $paginator->read();

In addition, it is possible to eg. to transform the keys, obviously must be reset beforehand.
This approach better covers all the options, but I recognize that some refinement is needed to get it out in absolute page numbers. I think this method is much better at covering the use between different paginators and less at risk of the accumulation of interfaces and methods.

I see. Questions:

On the third page, would you still pass -1 and 1 as $page? If so, that's not page number but a special value. In my case I have two separate methods for that and it's way cleaner about intent. How about making two separate methods?
Do you have a real case for more than a single parameter?

Do you have a real case for more than a single parameter?

RandomKeysetPaginator extends KeysetPaginator like Google's I'am Feeling Lucky search button. Although this is a sort order, it is technically provide by DateReader (eg DataReader->withRandomSeed(...)). In this case e.g. it works from a separate dataset, where the items are already randomized. However, this feature is limited by top of the paginator layers, and GridView can't even do it.
However, another purpose of the array I find useful is the black box. I don't need to know what parameters it produces and why, just work with it. I think this is such a closed part of the layer that there is no point in moving the meaning of the parameters to the higher layers.

On the third page, would you still pass -1 and 1 as $page? If so, that's not page number but a special value. In my case I have two separate methods for that and it's way cleaner about intent. How about making two separate methods?

This is a relative number of pages, but the methods also support absolute page numbers.
Basically I like the interface.
Would that be something like that?
$paginator = $paginator->withNextPageParameter($_GET[lastValue])->withPreviousParameter($_GET[firstValue]);

$nextUrl = ... lastValue=$paginator->getNextPageParameter()
$previousUrl = ... fistValue=$paginator->getPreviousPageParameter()

I started the planning with the level of use: layer view with N page link.

previous page, current, next page ([-1,0,1])
first page, previous page, current, next page, last page ([PHP_INT_MIN, -1, 0, 1, PHP_INT_MAX])
first page, N-2, N-1, current, N+1, N+2, last page ([PHP_INT_MIN, -2, -1, 0, 1, 2, PHP_INT_MAX])

I feel like I accept your interface proposal because it is really better as an interface. I suggest replacing the parameter to token.
interface PaginatorInterface
{
    public function read(): iterable;
    public function getNextPageToken(): ?string;
    public function getPreviousPageToken(): ?string;
    public function withNextPageToken(string $token): self;
    public function withPreviousPageToken(string $token): self;
    // or more useful (?string)
    // $paginator = $paginator->withNextPageParameter($_GET[lastValue] ?? null)->withPreviousParameter($_GET[firstValue] ?? null);
    public function withNextPageToken(?string $token): self;
    public function withPreviousPageToken(?string $token): self;
}

And another interface (I'm not convinced that all of them are based on the OffsetPaginator class.):
// This isolation for OffsetPaginator style.
interface PaginatorWithTotalPageNumberInterface {
    public function getTotalPageNumber(): int
    public function getDirectPageToken(): int
    public function withDirectPageToken(int): self;
   // or more useful (?int)
    public function withDirectPageToken(?int): self;
}

interface PaginatorInterface {
   public function withPageSize(int): self;
   ....
}

What do you think?
interface PaginatorInterface
{
    public function read(): iterable;
    public function getIsLastPage(): bool;
    public function getIsFirstPage(): bool;
    public function getNextPageToken(): ?string;
    public function getPreviousPageToken(): ?string;
    public function withNextPageToken(?string $token): self;
    public function withPreviousPageToken(?string $token): self;
    public function withPageSize(int): self;
}

I'm feeling lucky
Am I correct that "Am I correct that "I'm feeling lucky" thing is directing you to the first result from the whole result set? Then I don't get why there are any parameters or pagination needed at all... That is a custom data reader with a single "read()" method." thing is directing you to the first result from the whole result set? Then I don't get why there are any parameters or pagination needed at all... That is a custom data reader with a single "read()" method.
Interfaces

Glad we came to agreement that having separate methods is better than using magic number with multiple meanings as parameter.
Replacing parameter with a token sounds fine.
Planning OffsetPaginator interface later (if needed) is OK.
Interface above is fine. Let's introduce it :+1:

That is a custom data reader with a single read() method.

If we were to implement these two methods from DataReaderInterface?
public function withPageSize(int): self;
public function read(): iterable;

interface PaginatorInterface extends DataReaderInterface {
...
}

I don't think PaginatorInterface should extend DataReaderInterface. There's no need to manipulate limit directly in paginators. The interface above, as I said, looks fine.
As for I'm feeling lucky, implementation should be quite straightforward:
final class IAmFeelingLuckyDataReader implements DataReaderInterface
{
    private $dataReader;

    public function __construct(DataReaderInterface $dataReader)
    {
        $this->dataReader = $dataReader->withLimit(1);
    }
    public function withLimit(int $limit)
    {
        throw new NotSupportedException('I am feeling lucky has limit = 1 all the time.');
    }

    public function read(): iterable
    {
        return parent::read();
    }
}

@kamarton anything left in the issue that is not done yet?

No, closeable.


Widget type switches when switching between backends

When switching backends using the cell magic it switches to the scrubber widget for no apparent reason.
import numpy as np
import holoviews as hv

%load_ext holoviews.ipython
%output backend='bokeh'

%%output backend='matplotlib' holomap='widgets'
hv.HoloMap({i: hv.Curve(np.random.rand(10,2)) for i in range(10)})

The third cell always uses scrubber widgets the first time you execute it here whether you specify holomap='widgets' or not.

Ok, I'll look at this as soon as I have a chance.

This was fixed by #308. Closing.


spark-dev overwrites atom and apm commands on launch

Launching spark-dev (v0.0.19, not sure about earlier versions) causes the atom and apm commands (in /usr/local/bin on OSX) to be overwritten to point to spark-dev, instead of my Atom installation.
I use Atom for most of my normal development work, so this is extremely frustrating.

This has been fixed in #57 and will be available in next release planned for next week.


Flickr group issue / conflict with other module

I'm having two separate issues: one more with your module, the other may be out of bounds.
First, I'm unable to load Flickr group photos (https://www.flickr.com/groups/1059982@N21/pool/). My config code is:
module: "MMM-Wallpaper", psoition: "fullscreen_below", config: { source: "flickr-group:1059982@N21", slideInterval: 10*60*1000 }
All I get is a black background. If I change the source to "bing" images load just fine.
The second issue is when I also load MMM-NetworkConnection (https://github.com/slametps/MMM-NetworkConnection): it seems to override MMM-Wallpaper. I only get a black screen (no matter the source). When I comment out NetworkConnection module Wallpaper works. I'm just not smart enough to look at each of your sources to determine where the conflict is.
Thank you in advance, and thanks for developing and sharing your work!

For the flickr issue, you probably need to set flickrHighRes: false in your config.  Some time ago, they changed the way they generate the high-res urls, and I've never gotten around to figuring out how to fetch them now.
I really don't know how MMM-NetworkConnection could be interfering.  Does it show when you enable it, or is the screen completely black?  Perhaps it's an issue with an invalid config when you have them both?

Thanks for the quick reply. Turning off high-res worked! Thank you!
As for NetworkConnection: when I enable it and yours, no wallpapers load. It's just a black background, but all other modules work and show properly.

MMM-NetworkConnection depends on speedtest-net, which is patching the core node.js https.request method, and something about its patching isn't playing nicely with MMM-Wallpaper.

It looks like https://github.com/ddsol/speedtest.net doesn't depend on agent-base anymore, and agent-base doesn't even monkeypatch https.request anymore, so I think this is a bug for MMM-NetworkConnection to update their speedtest-net version

Okay - I see. Thank you for your help!

I added a package.json with a dependency on the request module, try updating to latest (you'll have to run npm install in the module directory now) and it should work with MMM-NetworkConnection now.


Change Bitcoin to Dogecoin in QA output strings

This is a tiny nit, possibly not worth it, but in reviewing QA debugging output or searching for it online, updating the coin name could remove a small amount of friction and confusion.

I've updated those places as well and scanned the qa/ directory for any other likely candidates. All that's left is uses of BITCOIN and bitcoind in the RPC test runner.
Renaming the environment variable feels more invasive than I'd like; what do you think?


[Proof] Get accumulator consistency proof from storage and implement verification

Motivation
With this change we will be able to get accumulator consistency proof from storage and verify it with root hashes.
It also moves the in-memory accumulator implementation from scratchpad to types so we are able to reuse the code.
Have you read the Contributing Guidelines on pull requests?
Y.
Test Plan
CI.

I'm leaning toward taking @msmouse's recommendation and going with another approach.


[labs/react] Fix use-controller.js not being exported from package.json

Fixes https://github.com/lit/lit/issues/2948

Very coo, very clean PRl! I don't have any issue submitting. Just need to confirm this part of the package is to be made public at the moment


Stop using keyid to match exchange for incoming message

Problem
What is being fixed?

Currently, SecureSessionHandle is used to match exchange for the incoming message in addition to exchange id and node id, but an exchange is not tied to a secure session in the spec. Before app send an message or create an new exchange, there is no way to know the keyid of a SecureSessionHandle up front, right now, we temporarily  hard coded the keyid to 0 to pass the exchange match check.

Fixes #4451

Change overview
Stop using keyid to match exchange for incoming message.
Testing
How was this tested? (at least one bullet point required)

./chip-echo-responder
./chip-echo-requester
confirm the echo test pass with this change.
./chip-im-responder
./chip-im-initiator
confirm the im test pass with this change.

Discussed with Boris and Martin, actions should be taken on the spec side first.

@turon could you confirm if the change in this PR aligned with your proposal in spec?

The new patch resolves my concern, but I still think comparing mSecureSession is the right way.
I'm considering remove all packetHeader and peerAddress args above transport layer, because it is not trust-able, and shouldn't be used above transport layer.

The new patch resolves my concern, but I still think comparing mSecureSession is the right way.
I'm considering remove all packetHeader and peerAddress arguments above transport layer, because it is not trust-able, and shouldn't be used above transport layer.

I agree.  Does this block this PR, or do we want to write some TODO tickets?
When messages come in the door, we have:

session ID -> by definition authenticated
source ID -> part of plaintext nonce
destination ID -> plaintext authenticated

Obviously session ID is the primary key we should be using to recover internal context.  Perhaps source ID and destination ID are also OK to pass to higher levels of the stack.  But allowing higher level code to key on anything else seems like a good way to inadvertently create security vulnerabilities.

The new patch resolves my concern, but I still think comparing mSecureSession is the right way.
I'm considering remove all packetHeader and peerAddress arguments above transport layer, because it is not trust-able, and shouldn't be used above transport layer.

I agree. Does this block this PR, or do we want to write some TODO tickets?
When messages come in the door, we have:

session ID -> by definition authenticated
source ID -> part of plaintext nonce
destination ID -> plaintext authenticated

Obviously session ID is the primary key we should be using to recover internal context. Perhaps source ID and destination ID are also OK to pass to higher levels of the stack. But allowing higher level code to key on anything else seems like a good way to inadvertently create security vulnerabilities.

The spec change(CHIP-Specifications/connectedhomeip-spec#3588) has specified how should we identify an exchange.
The Exchange is then identified by the tuple {Session ID, Initiator Fabric ID, Initiator Node ID, Exchange ID}.
The logic in the PR is implemented against the spec except comparing the Initiator Fabric ID (Fabric logic is not completely ready in transport layer, will add this logic later to fully comply with the spec).

When messages come in the door, we have:
session ID -> by definition authenticated
source ID -> part of plaintext nonce
destination ID -> plaintext authenticated

Just as a minor correction: for unicast encrypted messages, spec does not send a source id and destination id at all.  They are recoverable (and recovered) from the session id.

The new patch resolves my concern, but I still think comparing mSecureSession is the right way.
I'm considering remove all packetHeader and peerAddress arguments above transport layer, because it is not trust-able, and shouldn't be used above transport layer.

I agree. Does this block this PR, or do we want to write some TODO tickets?
When messages come in the door, we have:

session ID -> by definition authenticated
source ID -> part of plaintext nonce
destination ID -> plaintext authenticated

Obviously session ID is the primary key we should be using to recover internal context. Perhaps source ID and destination ID are also OK to pass to higher levels of the stack. But allowing higher level code to key on anything else seems like a good way to inadvertently create security vulnerabilities.

The spec change(CHIP-Specifications/connectedhomeip-spec#3588) has specified how should we identify an exchange.
The Exchange is then identified by the tuple {Session ID, Initiator Fabric ID, Initiator Node ID, Exchange ID}.
The logic in the PR is implemented against the spec except comparing the Initiator Fabric ID (Fabric logic is not completely ready in transport layer, will add this logic later to fully comply with the spec).

Yes, this needs to be addressed in (CHIP-Specifications/connectedhomeip-spec#3588) as well.  What's proposed there doesn't make sense.
The session ID is an abstraction that, from the perspective of a recipient, can map to a subject.  But the subject may or may not be a fabric+node identity.  In the case of a PASE session, it isn't.  If exchanges can exist over PASE, then keying on fabric+node identity doesn't work.

Hold until https://github.com/CHIP-Specifications/connectedhomeip-spec/pull/3588 is resolved

Removing SDK Discussion required, this doesn't require anything beyond the normal PR approval process.

@kghost  could you take another  look? The 'changes requested' seems to be very old (30 days ago) and we had many changes in since then.

I'm working with Michael for a solution on this PR.

I'm working with Michael for a solution on this PR.

Can we merge this though in the meantime?   It is conformant to https://github.com/CHIP-Specifications/connectedhomeip-spec/pull/3706.  I think we can introduce the followup work in other PRs.
This PR fixes a security problem by now properly associating PASE and CASE sessions to the local key ID instead of the peer key ID, so is useful on its own.

It looks fine to me. I'd like to rename SecureSessionHandle to SessionHandle

Thanks, that is already part of this change.


feat: add boost.io

add boost.io
need skip url stability check

Hello @bazelbuild/bcr-maintainers, modules without existing maintainers (boost.io) have been updated in this PR. Please review the changes.


Try/catch on prop set. TickLabels and svg/fig exports.

Hi,
I added a few things to the Plot class.

X/Y/ZTickLabels. I put this in primarily because it's a pain in MATLAB to get plots to display full decimal for integers (1.0 vs 1).

I enclosed getting the data in a try/catch block which makes it convenient to use with "non standard" plot types, like an errorbar plot. This way someone can use, for example, the MATLAB function errorbar then call Plot to make things look nice!

Added SVG and fig to exports. SVG mainly because Inkscape doesn't do well with fonts in eps files. Fig addition fixes #9.

Regards,
Hythem

Hi Hythem,
Thanks for the pull request. I'll have a look at it and merge it whenever I find some bandwidth.


Failed to load applications list, code 0

When I am trying to use to Dapp store in the desktop wallet I am getting the error:
Failed to load applications list, code 0
How do I fix this?
I am on Windows 11 insider edition but it didn't work on Windows 10 either.
Thanks for your time!

hello @sheeeev66 what version of the wallet do you use?


A small discrepancy in Play readme part

Having lazy val router: Router = wire[Routes] causes a type mismatch. I think you did mean having lazy val router: Router = wire[Router] in here.

Of course, thanks :)

I am deeply sorry, but could you revert that merge? I was not aware that Router is a trait, which can not be wired in here. Routes is the correct implementation of it. Due to messed up imports on my side, I did receive a compile error at that point.

sure, reverted. Thanks for checking anyway :)


Added a check to VMI validating webhook to not allow non-dns1123 comp…

…liant names for disks as the name is used for a container inside of the VM pod. Fixes issue #2161

What this PR does / why we need it:
Disk names can be used as container names inside the VMI pod. Without this validating webhook addition, the VMI will get created successfully but the pod will fail to get scheduled.
Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2161
Special notes for your reviewer:
Release Notes
NONE

@rmohr @SchSeba PR for issue we had discussion on a couple weeks back

Added a check for volume names too, weren't any existing tests for volumes so I didn't add one there though.

I can definitely squash this commit if you guys would like as well. Kind of a cowboy committer here.

somehow the prob job history for this PR is gone. Let's wait for new test results...

somehow the prob job history for this PR is gone. Let's wait for new test results...

ugh

somehow the prob job history for this PR is gone. Let's wait for new test results...

Ah ok, they never ran :)
/ok-to-test

somehow the prob job history for this PR is gone. Let's wait for new test results...

Ah ok, they never ran :)

Ok, that explains it, totally missed that :laughing:

/retest


Please help

I need a version of 3RVX wich contains both: Volume amount AND Media key support.
I don't understand why the volume amount setting from 2.5 was wiped out.
Maybe someone can merge these settings for me oder explain how to do.

Does Volume Amount mean for you how many steps until 100%? Or increase of volume per Volume Key?
On one side it is controlled by the skin with the entry:
<meterMax>50</meterMax>
The number gives amount of steps.
I´m using 'Hybrid-X Cold' because it is visually the best to adjust.
Have you tried 'Audio taper' (Log-Lautstärkeregler), to get the desired result?


Display events

Allows the DisplayDevice class to act as an observable system, that generates an event for resizing.
Need
As things stand, certain systems in Rendering Land (like Cameras and DisplayResolutionDependentFBOManagers) should react to resizing events, but due to lack of the events, they either react late or they have to manually check Display.isResized(). For instance, in the screenshot attached, the world is rendered incorrectly (squashed) because perspectiveCamera does not immediately register the change.
With these events, it should also be possible to optimize certain parts of the code like not regenerating the projectionMatrix every frame, which currently happens.

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

Hooray Jenkins reported success with all tests good!

As I tested this, I noticed that resizing the window while the menu is visible still generates squashing and stretching, which is only corrected when you get back into the game. Can this be avoided @vampcat? Can the view of the 3d scene be updated even when the menu is present?

@emanuele3d That was definitely a mistake on my part.
When I was initially testing the PR (and upto 2-3 commits ago) the resizing worked perfectly, even when paused. However, I am working on a Linux machine atm, and apart from the horrendeous performance making me want to kill myself, Terasology seems to be bugged on it (character always keeps moving, forcing re-gerenation of matrices every frame). Due to that, a change escaped my notice, and hence the weird behgaviour for you.
With the last commit, it should be fixed.

Hooray Jenkins reported success with all tests good!

Yoink! Grabbing this with a decent level of confidence that it is ready to go ... :-)
Also, woo for if (!screenGrabber.isNotTakingScreenshot()) {
While testing I noticed after changing graphic settings to minimal + setting screenshot size to "quarter size" that taking a screenshot would provoke an error in the log that didn't seem to actually cause any problems:
20:46:25.140 [main] ERROR org.terasology.rendering.opengl.FBO - FrameBuffer: engine:fbo.sceneReflectiveRefractive, has caused a GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT exception
Switching to Ultra + double size screenies seemed to cause the same issue. Switching to medium + normal size made it go away. So maybe it is specific to taking non-standard sized screenies.
All actual screenies seemingly came out fine and no other issues noted. Merging! :-)

Thank you for the merge @Cervator, even though I would have left the PR open a little longer given the GL_FRAMEBUFFER_INCOMPLETE_DIMENSIONS_EXT error.
@vampcat, were you able to reproduce?


Observe should state that Trigger<_> must be first arg

How can Bevy's documentation be improved?
Observe
There is a limitation that Trigger<_> must be the first arg.
So currently when doing:
App::new().observe(f);

then:
/// works
fn foo(trigger: Trigger<Foo>, mut commands: Commands) {}

/// does not
fn bar(mut commands: Commands, trigger: Trigger<Foo>) {}

this should be documented.

We should ensure that the same note is present for exclusive and pipe systems while we're at it.


Error message could be better if home directory isn't found (because it doesn't exist)

$ cd libc-test && cargo run
error: An unknown error occurred

To learn more, run the command again with --verbose.

What does it mean? :)
$ cd libc-test && cargo run --verbose
error: Permission denied (os error 13)

Errr... yes? Where?
strace reveals that it's because it can't find the user's home directory (because there's none in the chroot). Setting the environment variable "CARGO_HOME" or creating a ".cargo" directory makes the error go away.
But it would be nice if it also printed the filename.

This seems like a Cargo issue? Can you use strace to figure it out?

It? What do you mean?
It's working now. This was just a heads-up about it so that the error message doesn't stay that mysterious forever.

@daym it seems that a phantom comment may have come in? Is it working now? I was thinking of using strace to figure out the filename.

I'm going to close this as I think this isn't an issue for libc


⚠ Bofrost.at has degraded performance

In fe727e7, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2865 ms

Resolved: Bofrost.at performance has improved in 4e192df after 51 minutes.


eStark backend via eigen-zkvm's starky

WARNING: only merge this after https://github.com/0xEigenLabs/eigen-zkvm/pull/94 has been merged. Merged.

/usr/bin/time -v output of a full run, from rust to proof:
	Command being timed: "./target/release/powdr rust --field gl --prove-with estark riscv/tests/riscv_data/vec_median -i 5,11,15,75,6,5,1,4,7,3,2,9,2 -o tmp"
	User time (seconds): 3037.97
	System time (seconds): 40.50
	Percent of CPU this job got: 573%
	Elapsed (wall clock) time (h:mm:ss or m:ss): 8:56.58
	Average shared text size (kbytes): 0
	Average unshared data size (kbytes): 0
	Average stack size (kbytes): 0
	Average total size (kbytes): 0
	Maximum resident set size (kbytes): 30918820
	Average resident set size (kbytes): 0
	Major (requiring I/O) page faults: 17168
	Minor (reclaiming a frame) page faults: 24416794
	Voluntary context switches: 25376
	Involuntary context switches: 88996
	Swaps: 0
	File system inputs: 1061608
	File system outputs: 2127928
	Socket messages sent: 0
	Socket messages received: 0
	Signals delivered: 0
	Page size (bytes): 4096
	Exit status: 0

Two things missing in this PR:

[ ] Use it for proofs the same way Halo2 is already used for proofs in the compiler end to end tests
[ ] We need to inject the first_step column if it doesn't already exist in the given PIL. This will be needed for the item above as well.

https://docs.powdr.org/backends/
This should be updated

https://docs.powdr.org/backends/
This should be updated

follow up PR?

I personally try to keep the book in sync with the PRs as I think it's less work than accumulating a bunch of undocumented stuff and documenting it later. Also I have not been seeing a lot of such standalone book update PRs.

I personally try to keep the book in sync with the PRs as I think it's less work than accumulating a bunch of undocumented stuff and documenting it later. Also I have not been seeing a lot of such standalone book update PRs.

updated


Icons not showing on Build Tasks invoked with the shortcut ctrl+shift+B

Issue Type: Bug
Create at least two Task assigned to the build group. Add an icon to this task.
example :
        {
            "label": "Example command 1",
            "type": "shell",
            "command": "echo hello1",
            "group": "build",
            "icon": {
                "id": "package"
            }
        },
        {
            "label": "Example command 2",
            "type": "shell",
            "command": "echo hello2",
            "group": "build",
            "icon": {
                "id": "beaker"
            }
        }

Press ctrl + shift + B to launch the build Task. A dropdown opens and lists the two tasks. Neither of the tasks has its icons displayed on the left of their name.

Expected results :
The icons show on the left of the name, same as when you use the Task: Run task command.

VS Code version: Code 1.69.0 (92d25e35d9bf1a6b16f7d0758f25d48ace11e5b9, 2022-07-07T05:28:36.503Z)
OS version: Windows_NT x64 10.0.17763
Restricted Mode: No
Remote OS version: Linux x64 3.10.0-1160.66.1.el7.x86_64

/duplicate #153699

Thanks for creating this issue! We figured it's covering the same as another one we already have. Thus, we closed this one as a duplicate. You can search for similar existing issues. See also our issue reporting guidelines.
Happy Coding!


bam_to_h5.R fails on Eddie when run inside screen

For me as of May 2021, this works:
source set-riboviz-env.sh
cd riboviz/riboviz
nextflow prep_riboviz.nf -params-file vignette/vignette_config.yaml

This doesn't:
source set-riboviz-env.sh
cd riboviz/riboviz
screen
source activate riboviz
nextflow prep_riboviz.nf -params-file vignette/vignette_config.yaml

The pipeline fails at bam_to_h5.R with the error message:
/exports/igmm/software/pkg/el7/apps/R/3.6.3/lib64/R/bin/exec/R: error while loading shared libraries: libgfortran.so.4: cannot open shared object file: No such file or directory

so it is something to do with finding R libraries from inside screen.
Is it worth adding a line to docs/user/run-on-eddie.md saying "don't try to run riboviz inside screen"? Or not?

I think this is a similar problem to https://stackoverflow.com/questions/50591901/screen-inside-the-conda-environment-doesnt-work/53535739
We need to start a screen session before load anaconda and riboviz environment, and this works:
screen
source set-riboviz-env.sh
cd riboviz/riboviz
source activate riboviz     # if you didn't activate riboviz in set-riboviz-env.sh
nextflow prep_riboviz.nf -params-file vignette/vignette_config.yaml

It looks like this is a documentation issue that could be added somewhere.
@mikej888 @FlicAnderson do you think we need to add to our documentation, or just close without fixing?

Commit 5b5c714 has a Troubleshooting section added to the end of docs/user/run-on-eddie.md with @ewallace issue and @canuse workaround.

@mikej888: can we close this, is it done?

@kavousan I think so as we've documented the issue and a workaround.


Payments - One error message appeared under the input when typed 31/01/0001

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Issue found when executing PR https://github.com/Expensify/App/pull/15978
Action Performed:

Go to staging.new.expensify.com
Login with expensifail account
Navigate to /settings/payments/enable-payments page
In Date of Birth input type 31/01/0001
Verify that two errors appeared under the input

Expected Result:
Two error messages appeared under the input when typed 31/01/0001
Actual Result:
One error message appeared under the input when typed 31/01/0001
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: <IP_ADDRESS>
Reproducible in staging?: Yes
Reproducible in production?: Yes
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/93399543/227733506-ff679c7d-434f-4dc2-8f2f-4897f77c6d6c.mp4

Expensify/Expensify Issue URL:
Issue reported by: Applause - Internal Team
Slack conversation:
View all open jobs on GitHub

Triggered auto assignment to @miljakljajic (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

The test steps in the PR are outdated because of this comment
This is working as expected. Closing issue.


fix: ensure project_filter_list is read from the global project

Summary
The setting "project_filter_list" is not documented as mandatory for regional lacework_gcp_agentless_scanning modules, only in the global module.  But, prior to this PR, it actually is mandatory - scanning will not work in the regions if it is missing.
This PR remedies the situation by ensuring that this setting is read from the global module and passed to the regional ones.
How did you test this change?

[x] unit tests in CI
[x] tested with Support's help.

Issue
N/A, reported internally.

Great call. I have a similar issue in Azure. Will fix it


About panel and l10n checks

added new keys in labels.properties for AboutPanel
refactored l10n class
added l10n check to CI test, will trigger a job fail & report if any translation is missing

a failed job for this PR is normal (AboutPanel translations are missing for every language except English), howewer I'm havin doubts about this line in the yaml workflow file:
token: ${{ secrets.RELEASE_TOKEN }}

if your Workflow permissions aren't set on **
Read and write** you may need to restore the token

The PR looks good for merge if you don't have anything else to add let me know.

a failed job for this PR is normal (AboutPanel translations are missing for every language except English), howewer I'm havin doubts about this line in the yaml workflow file:
token: ${{ secrets.RELEASE_TOKEN }}

if your Workflow permissions aren't set on Read and write you may need to restore the token, otherwise it isn't needed

The PR looks good for merge if you don't have anything else to add let me know. I will remove that line and see if it can build a release without issue.

I have no further commits for this PR

trying to advertise the new l10n support on the social networks ...
https://mastodon.social/@tobozo/112290798173744999
https://twitter.com/TobozoTagada/status/1780846712086569342
... and realized the ReadMe isn't obvious about how translators should peer-review, do you want a PR for this change?
Issues and Contributions
Feel free to report any issues. Translation contributions are welcome if you find the current translations to be wrong, not adequate or you want to add support for another language.

I will update the readme. I will look at your repo to see whats up :)


Allow headers to be replaced

I tried sending an email formatted as HTML, by including an extra header:
additionalHeaders: [(header: "Content-Type", value: "text/html")]

However, we get this error:
554 Transaction failed: Duplicate header 'Content-Type'.

It looks like SMTPSocket is still sending Content-Type: text/plain along with the new header, which is invalid.
Is there a way we can update the API to overwrite headers rather than just appending them?  Or, for my specific use case, an alternative method for sending HTML emails?

We will change the APIs to overwrite headers instead of append them. This is how it originally was, but was changed for safety. But now I see it's necessary to give users this power.
One alternative for sending HTML emails is creating an Attachment of type HTML and attach that along with the Mail you are sending.

@quanvo87 it looks like there's still one lingering issue:  in DataSender.swift there's a string that seems to get prepended to every message:
static let plainTextHeader = "CONTENT-TYPE: text/plain; charset=utf-8\(CRLF)CONTENT-TRANSFER-ENCODING: 7bit\(CRLF)CONTENT-DISPOSITION: inline\(CRLF)"

Since we're still hard-coding Content-Type, there's a duplicate and it gets rejected by the server.
Are these headers actually necessary? I wonder if we could add them to the main headers dictionary prior to sending (if the user hasn't set them already), that way we avoid all conflicts.

It seems that header was not necessary. In the master branch now, it is removed completely. Can you see if that solved the issue?
Also, was sending the HTML as an attachment not an option?

Hey @quanvo87, we'll need a new release to test it out.  However - I made the same changes on my local revision but it doesn't seem to work... the email sends, but the message body is empty. Not sure why.

Could you post the email and any attachments you are sending?

@quanvo87 here's an example:
// Create users
let toUser = SwiftSMTP.User(email: toAddress)
let fromUser = SwiftSMTP.User(email: fromAddress)

// Create SSL credentials
let ssl = SSL(withChainFilePath: "/my/cert/path",
              withPassword: "myCertPassword",
              usingSelfSignedCerts: true)

// Create SMTP instance
let smtp = SMTP(hostname: "email-smtp.us-west-2.amazonaws.com",
                   email: "myAWSUsername",
                   password: "myAWSPassword",
                   port: 587,
                   ssl: ssl)

// Create email object
let body = "This is a test email.

It is formatted as HTML

It should work."
let email = SwiftSMTP.Mail(from: fromUser, to: [toUser],
                           subject: "This is a test", text: body,
                           additionalHeaders: [
                            "Content-Type" : "text/html; charset=utf-8"
                           ])

// Send email via AWS
smtp.send(email) { (error) in
    // ...
}

The message comes through with the correct subject, but a completely empty body.
It looks like plainTextHeader might have been important... if I keep plainTextHeader, but remove the Content-Type portion, everything works correctly.

I ran this code on the latest release, and the email text was correctly formatted (I connected with my Gmail account). However when I removed the Content-Type header, the body was not blank, but also not correctly formatted.
So you're right, the plain text portion of an email should have a header in by default, but it should be text/html, not text/plain.
I don't know why the body was empty, and I couldn't reproduce that. Possibly an issue with AWS?

@quanvo87 I've switched over to my Gmail account to make sure it wasn't an AWS issue, but the results are the same. However, I think I've pinpointed part of the issue...
The email body that I was actually sending is slightly different.  Try running the same test with this body:
"This is a test HTML message.

Here is some more text:
Hello."
Empty message.
But, if you remove the colon:
"This is a test HTML message.

Here is some more text
Hello."
It renders correctly.
  But, if you prepend plainTextHeader to the message like how it used to be (but remove the Content-Type portion while keeping your additional text/html header), the message renders correctly. For example:
body = "CONTENT-TRANSFER-ENCODING: 7bit\(CRLF)CONTENT-DISPOSITION: inline\(CRLF)\(CRLF)This is a test HTML message.

Here is some more text:
Hello."
So, it looks like the email servers require there to be some sort of header directly attached to the message body; however, I'm not sure what the spec actually calls for.   The colon caused the body to be blank. I have no idea why.
We have a PR open that addresses this. In summary,

For plain text content of an email, these default headers are added:

CONTENT-TYPE: text/html; charset=utf-8
CONTENT-TRANSFER-ENCODING: 7bit
CONTENT-DISPOSITION: inline

These headers can be overwritten by the user with the additionalHeaders dictionary

This is usually not recommended as any headers in additionalHeaders are applied to the entire email, so multi content-type emails will be rendered incorrectly. This will be noted in the docs.

Additional changes to the other content types (mixed, alternative, related) will also be made so that the default CONTENT-TYPE header can be overwritten as well (but again, not recommended)

By the way, when I sent:
body = "CONTENT-TRANSFER-ENCODING: 7bit\(CRLF)CONTENT-DISPOSITION: inline\(CRLF)\(CRLF)This is a test HTML message.

Here is some more text:
Hello."
My email rendered like this:
`CONTENT-TRANSFER-ENCODING: 7bit CONTENT-DISPOSITION: inline This is a test HTML message.
Here is some more text:
Hello.`
Is this what you saw as well?

@quanvo87 great to hear!  I glanced over the PR; it looks like this should work nicely.
With the body mentioned above, did you still set additionalHeaders to ["Content-Type" : "text/html; charset=utf-8"]?  When I do that, it renders correctly:
This is a test HTML message.

Here is some more text:
Hello.

@quanvo87 the new release seems to mostly work - I'm now able to send HTML emails.
However, attempting to change the Content-Type gives us this error:
554 Transaction failed: Duplicate header 'Content-Type'

This happens for example if I try to send a plaintext email and set additionalHeaders to this:
additionalHeaders: ["Content-Type" : "text/plain; charset=utf-8"]

I'm not 100% sure, but it looks like we're sending headers in 2 places:  in the actual request headers, and prepended to the message body.  Since Content-Type is being sent in both of those places, if they don't match up we get an error.

What should be happening is the request headers and any additionalHeaders are sent first, and these headers apply to the entire email. The rest of the email consists of subsections, which will have their own headers based on what type content they are (plaintext, mixed, alternative, related).
For plaintext, we check for Content-Type, Content-Transfer-Encoding, and Content-Disposition in the additionalHeaders. If they are not present, we use default values. So yes, headers of these types will be used twice, once for the overall email, and again for the "plaintext" section of the email. This seems to be causing problems for some servers (Gmail seems to be ok with duplicate headers), so the current implementation will not work.
Maybe any headers in additionalHeaders should not be applied to the entire section of the email, but to each subsection. This would break things where additionalHeaders are misused (but maybe we can check for this, and throw away invalid headers). We still need to think of the full implications of this.
What is the reason to set the Content-Type to text/plain for a plaintext header? It should send correctly without doing so.

@quanvo87 I have no reason to use test/plain myself, but was just testing stuff. However, the same error occurs if you add Content-Type: text/html; charset=utf-8 to additionalHeaders.

This is addressed in the latest release https://github.com/IBM-Swift/Swift-SMTP/releases/tag/v1.0.8

@quanvo87 Everything seems to work great. Thanks!

@collinhundley Thanks for your feedback! There's surely more improvements to be made and please don't hesitate to open up another issue

Issue moved to IBM-Swift/Kitura #1131 via ZenHub


Added note about arrays

Added note about arrays being returned as JSON.

This occurs because of the resent header parser change in core if I'm not mistaken, correct?

@fl0w I am sorry I got some trouble understanding the English in that sentence, could you rephrase it please? What header parser change in core? (is that something that happened in Koa? or something I don't understand?)
If it makes it any more clear, I wrote an issue here that explains why it happens: https://github.com/koajs/koa/issues/965


All messages go to spam...

Is there an issue? This form otherwise works wonderfully! here is my config:
 [
        'prefix' => '[Contact Form]'
    ],
    'emails' => [
        'to'   =><EMAIL_ADDRESS>        'from' => ''
    ],
    'messages' => [
        'error'   => 'There was an error sending, please try again later.',
        'success' => 'Your message has been sent successfully.'
    ],
    'fields' => [
        'name'     => 'Name',
        'email'    => 'Email',
        'phone'    => 'Phone',
        'subject'  => 'Subject',
        'message'  => 'Message',
        'btn-send' => 'Send'
    ]
];

Hello @henry7720,
In the 'from' field add an email with the same domain where the form is hosted. This prevents emails from falling into SPAM.


Is there are reason we can't convert wikilinks to markdown links?

I was wondering why there is a command Convert URL links to WikiLinks in selection but not the other way round?
It would be great to be able to convert an (Obsidian) wikilink to plain markdown (with an option to set an absolute or a relative path).

v2.6.0 has supportted this feature, please have a try!
Free feel to reopen this issue if something is wrong.

Thank you, works like a charm!

I'm encountering issues with the "Convert WikiLinks to plain Markdown links" feature affecting attachments and the handling of spaces in WikiLinks.
Description:

Attachments receive ".md" extension: When converting WikiLinks for attachments, a ".md" extension is incorrectly added (e.g., ![[image.png]] becomes  instead of ).

Inconsistent handling of spaces in WikiLinks: WikiLinks to notes handle spaces correctly by replacing them with %20 (e.g., [[Example Note]] converts to Example Note), but attachments with spaces break during conversion because the spaces remain unescaped (e.g., ![[my image.png]] converts to ![my image](my image.png) instead of ).

Steps to reproduce:
Use the "Convert WikiLinks to plain Markdown links" feature on:

A WikiLink to an attachment (e.g., ![[image.png]]).
A WikiLink to an attachment or note containing spaces (e.g., ![[my image.png]] or [[Example Note]]).

Update: Found https://github.com/mii-key/obsidian-links which works great for my use case.


Final sprint checkin + disclaimer

Hi everyone,
Here's your friendly reminder to keep track of the Wiki! You can cite your sources and keep track of your progress on their respective pages, and it would also be great if @lainecedes and/or @Bryan-OF could update the Design & Styling page with your design. So far I've also documented our weekly progress as a team, but an extra set of eyes making sure I'm not missing anything is never too much.
By the way, @Bryan-OF, you've used my 'matchingresult' code for your song page and its styling, right? I've actually edited that page and its CSS on my branch, so if we want to avoid conflicts when merging to main later on, the best practice for you would be:

Creating a new 'section' in style.css for your 'Liking' feature (note how I've distinguished my CSS from the rest with a large heading);
renaming the id's and classes in your EJS;
copying/pasting my CSS the way it is in your branch now to your section with the appropriate class names and id's.

That way, the styling of your page won't be affected by my changes, as you've made it your own :)
Lastly, if anyone finds themselves struggling with converting to controllers, you can join my Teams call with Jarno this weekend!

We're all working on our areas of the Wiki, so I'm closing this issue.


Rearranging Graph Node crosses connectors

Godot version
4.0.dev 50bb1846d0f16fae929b2ae99824dbf90be25c92
System information
Win 11, Vulkan, Nvidia 3000
Issue description
Create a blend tree graph node graph like the photo and there are crossings which shouldn't exist.
Steps to reproduce

Create graph node in blend tree
Rearrange the graph in this structure
There will be crossings.

Minimal reproduction project
I don't have one prepared, but poke me if you want one or want to upload one.

@theoway

I'm not sure what you mean by stating that this shouldn't happen. They cannot not cross in this case, so what do you propose should happen? Also, shouldn't this be a proposal?

This is a bug because it is possible to create an arrangement of this so that they don't cross. I made this bug a while ago to not forget about.
I agree it's also a feature enhancement too.


FAISS documenting filtering is broken

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.

Example Code
The issue is in langchain/libs/community/langchain_community/vectorstores/faiss.py at line 333
            if filter is not None and filter_func(doc.metadata):
                docs.append((doc, scores[0][j]))
            else:
                docs.append((doc, scores[0][j]))

If there's a filter, this will always add a document whether or not the function succeeds or fails (confirmed with AI: https://chat.openai.com/share/1b68d90b-ed4d-4e7d-9aff-9195acf18f96)
It should be something like:
            if filter is None:
                docs.append((doc, scores[0][j]))
            elif filter_func(doc.metadata):
                docs.append((doc, scores[0][j]))

Error Message and Stack Trace (if applicable)
No response
Description

using FAISS with a filter just basically ignores the filter

System Info
langchain==0.1.5
langchain-community==0.0.17
langchain-core==0.1.18
langchain-mistralai==0.0.3

I believe this is fixed in v0.1.7, line 336 here
if filter is not None:
    if filter_func(doc.metadata):
        docs.append((doc, scores[0][j]))
else:
    docs.append((doc, scores[0][j]))

Functionally it looks good. A cleaner approach might be:

        ## around line 308, define an always True filter_func before the rest of the code
        filter_func = lambda metadata: True
        if filter is not None:
            if isinstance(filter, dict):
            ...

Then at line 336:
            if filter_func(doc.metadata):
               docs.append((doc, scores[0][j]))


Cant start any consul cluster

I can start all examples/ at local but not working in my server (a Virtual Machine), consul can't join with other
I tried to run all command in docker-alpine/alpine-consul/root/usr/bin/ in consul-container
dig +short ${1:-$CONSUL_SERVER_NAME} in  container-find file not working, that make script consul-join, consul-available cant run  (and in this container, i cant ping to any website. )

Yeah, I've had dig not work before. Usually host or getent or something similar will. You should find something that works and replace that script with that works in your containers on the VM. Then, everything should be fine.
Also, have you considered moving to Kubernetes? There is no need for Consul in Kubernetes.

tks you! I run docker-compose first and run a script after that. The script show list ip of consul from the container outside ( base on docker-inspect) and export to text file. The text file is used to volumes from outside to /root/usr/bin/ inside container.  I changed container-find file -> read the text file.
This my way. I cant show list ip from inside container with dig, getent, host, ...  at my server (Debian Virtual Machine).
Can you help me another better way! Tks you!

when i changed dig to getent ahosts ${1:-$CONSUL_SERVER_NAME} | awk '{print $1}' | awk 'NR % 2'  at local
i receive list ip same dig
/ # getent ahosts ${1:-$CONSUL_SERVER_NAME} | awk '{print $1}' | awk 'NR % 2'
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
/ # dig +short ${1:-$CONSUL_SERVER_NAME}
<IP_ADDRESS>
<IP_ADDRESS>
<IP_ADDRESS>
/ #

BUT Consul NOT working
e, not adding Raft peer.

    2018/05/17 01:22:35 [INFO] consul: member 'e07b1d5ff119' joined, marking health alive

    2018/05/17 01:22:35 [ERR] consul: 'f0cf753f5b09' and 'e0d5eecc154a' are both in bootstrap mode. Only one node should be in bootstrap mode, not adding Raft peer.

    2018/05/17 01:22:35 [INFO] consul: member 'f0cf753f5b09' joined, marking health alive

    2018/05/17 01:22:35 [INFO] consul: New leader elected: 97368f1f55c0

    2018/05/17 01:22:36 [INFO] consul: New leader elected: f0cf753f5b09

    2018/05/17 01:22:36 [INFO] consul: New leader elected: f782fa5e353f

    2018/05/17 01:22:36 [INFO] consul: New leader elected: e07b1d5ff119

    2018/05/17 01:23:35 [ERR] consul: 'f782fa5e353f' and '97368f1f55c0' are both in bootstrap mode. Only one node should be in bootstrap mode, not adding Raft peer.

    2018/05/17 01:23:35 [ERR] consul: 'e07b1d5ff119' and 'e0d5eecc154a' are both in bootstrap mode. Only one node should be in bootstrap mode, not adding Raft peer.

    2018/05/17 01:23:35 [ERR] consul: 'f0cf753f5b09' and 'e0d5eecc154a' are both in bootstrap mode. Only one node should be in bootstrap mode, not adding Raft peer.

Are you familiar with how Consul works?
You could try setting CONSUL_BOOTSTRAP_DEBUG to true and then review the log file at /var/log/consul-bootstrap/consul-bootstrap.log. This documents decisions made by the custom Consul bootstrapping functionality within these containers.
Also, have a read of https://github.com/smebberson/docker-alpine#consul-and-service-discovery and review the example. See if you can get that to work.


[benchmark] Janitor Duty: Sweep Quadratic

To enable robust performance measurements by minimizing the accumulated error, this splits the composite tests from DictionatyCopy and DictionaryFilter into smaller individual benchmarks by dictionary size. These benchmarks guard against the quadratic behavior (SR-3268). Also removes the older disabled HashQuadratic benchmark that was covering the same issue.

@swift-ci please benchmark

Build failed before running benchmark.

@swift-ci please benchmark

@swift-ci please benchmark

Performance: -O

TEST
OLD
NEW
DELTA
RATIO

Regression

SetSubtractingInt100
151
167
+10.6%
0.90x (?)

ObjectiveCBridgeFromNSDictionaryAnyObjectToString
74000
80500
+8.8%
0.92x (?)

StringHashing_ascii
39
42
+7.7%
0.93x (?)

Improvement

ObjectiveCBridgeStubFromNSDateRef
5050
4190
-17.0%
1.21x (?)

ObjectiveCBridgeStubFromNSString
1020
903
-11.5%
1.13x (?)

ObjectiveCBridgeStubFromArrayOfNSString2
3730
3410
-8.6%
1.09x (?)

DictionaryRemove
3280
3030
-7.6%
1.08x (?)

Added

Dict.CopyKeyValue.16k
1137
1529
1268
—

Dict.CopyKeyValue.20k
3644
3717
3679
—

Dict.CopyKeyValue.24k
1252
1252
1252
—

Dict.CopyKeyValue.28k
1863
2376
2034
—

Dict.FilterAllMatch.16k
711
711
711
—

Dict.FilterAllMatch.20k
849
851
850
—

Dict.FilterAllMatch.24k
843
861
853
—

Dict.FilterAllMatch.28k
1333
1337
1335
—

Removed

DictionaryCopy
53497
58134
55079
—

DictionaryFilter
44472
44768
44581
—

Code size: -O

TEST
OLD
NEW
DELTA
RATIO

Regression

DictionaryCopy.o
7885
10182
+29.1%
0.77x

Performance: -Osize

TEST
OLD
NEW
DELTA
RATIO

Regression

SetIsSubsetBox0
362
462
+27.6%
0.78x

ObjectiveCBridgeFromNSSetAnyObjectToStringForced
96000
110500
+15.1%
0.87x (?)

SetSubtractingInt100
159
183
+15.1%
0.87x

ObjectiveCBridgeFromNSDictionaryAnyObject
40300
45700
+13.4%
0.88x (?)

SetSubtractingInt0
73
81
+11.0%
0.90x (?)

ObjectiveCBridgeFromNSSetAnyObject
53400
59200
+10.9%
0.90x (?)

ObjectiveCBridgeFromNSDictionaryAnyObjectToString
73500
80500
+9.5%
0.91x (?)

ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced
80000
87500
+9.4%
0.91x (?)

SetSubtractingInt25
101
110
+8.9%
0.92x (?)

SetSymmetricDifferenceInt100
220
239
+8.6%
0.92x (?)

ObjectiveCBridgeStubNSDateRefAccess
343
371
+8.2%
0.92x (?)

Improvement

Set.isDisjoint.Box25
713
610
-14.4%
1.17x

DictionaryRemove
5790
5340
-7.8%
1.08x (?)

Added

Dict.CopyKeyValue.16k
1695
2069
1820
—

Dict.CopyKeyValue.20k
4302
4365
4329
—

Dict.CopyKeyValue.24k
2057
2074
2067
—

Dict.CopyKeyValue.28k
2920
3426
3102
—

Dict.FilterAllMatch.16k
733
734
734
—

Dict.FilterAllMatch.20k
871
873
872
—

Dict.FilterAllMatch.24k
871
888
881
—

Dict.FilterAllMatch.28k
1380
1382
1381
—

Removed

DictionaryCopy
73998
79288
75766
—

DictionaryFilter
46360
46391
46373
—

Code size: -Osize

TEST
OLD
NEW
DELTA
RATIO

Regression

DictionaryCopy.o
6177
7958
+28.8%
0.78x

Performance: -Onone

TEST
OLD
NEW
DELTA
RATIO

Regression

SetIsSubsetBox0
879
1715
+95.1%
0.51x

SetIsSubsetInt0
633
733
+15.8%
0.86x (?)

Improvement

Set.isDisjoint.Box25
2611
2040
-21.9%
1.28x

ObjectiveCBridgeStubFromNSDateRef
5480
4510
-17.7%
1.22x (?)

DictionarySwapOfObjects
22200
18800
-15.3%
1.18x (?)

DictionarySwapAtOfObjects
17600
14920
-15.2%
1.18x (?)

ObjectiveCBridgeStubFromNSDate
7470
6450
-13.7%
1.16x (?)

SetExclusiveOr_OfObjects
37210
33090
-11.1%
1.12x

SetSymmetricDifferenceBox0
3721
3311
-11.0%
1.12x

DictionarySwap
4780
4352
-9.0%
1.10x (?)

SetUnion_OfObjects
25940
23910
-7.8%
1.08x (?)

SetUnionBox0
2594
2392
-7.8%
1.08x (?)

Added

Dict.CopyKeyValue.16k
6090
6692
6294
—

Dict.CopyKeyValue.20k
26754
27239
26962
—

Dict.CopyKeyValue.24k
6917
7033
6956
—

Dict.CopyKeyValue.28k
8536
9281
8821
—

Dict.FilterAllMatch.16k
4733
4862
4777
—

Dict.FilterAllMatch.20k
24734
24899
24808
—

Dict.FilterAllMatch.24k
4824
4901
4853
—

Dict.FilterAllMatch.28k
6166
6198
6178
—

Removed

DictionaryCopy
266292
271700
268214
—

DictionaryFilter
214044
216822
214972
—

✅
Benchmark Check Report

⚠⏱
Dict.FilterAllMatch.28k execution took at least 1333 μs.  Decrease the workload of Dict.FilterAllMatch.28k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.FilterAllMatch.28k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.28k mem_pages [i1, i2]: min=[757, 756] 𝚫=1 R=[39, 41]

⚠⏱
Dict.CopyKeyValue.20k execution took at least 3640 μs.  Decrease the workload of Dict.CopyKeyValue.20k by a factor of 4 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.20k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.20k mem_pages [i1, i2]: min=[368, 368] 𝚫=0 R=[78, 78]

⚠⏱
Dict.CopyKeyValue.16k execution took at least 1119 μs.  Decrease the workload of Dict.CopyKeyValue.16k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.16k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.16k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[39, 40]

⚠⏱
Dict.CopyKeyValue.28k execution took at least 1860 μs.  Decrease the workload of Dict.CopyKeyValue.28k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.28k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.28k mem_pages [i1, i2]: min=[756, 756] 𝚫=0 R=[39, 0]

⚠Ⓜ
Dict.FilterAllMatch.24k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.24k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[39, 0]

⚠⏱
Dict.CopyKeyValue.24k execution took at least 1247 μs.  Decrease the workload of Dict.CopyKeyValue.24k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.24k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.24k mem_pages [i1, i2]: min=[368, 368] 𝚫=0 R=[39, 39]

⚠Ⓜ
Dict.FilterAllMatch.20k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.20k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[39, 39]

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

@swift-ci benchmark

@swift-ci benchmark

@swift-ci please smoke test

@swift-ci please smoke test

@swift-ci please smoke test

Performance: -O

TEST
OLD
NEW
DELTA
RATIO

Regression

DataAppendDataLargeToLarge
38600
51200
+32.6%
0.75x (?)

Improvement

StringBuilderLong
1370
1230
-10.2%
1.11x (?)

Added

Dict.CopyKeyValue.16k
990
1387
1123
—

Dict.CopyKeyValue.20k
1106
1106
1106
—

Dict.CopyKeyValue.24k
1245
1246
1246
—

Dict.CopyKeyValue.28k
1893
2403
2064
—

Dict.FilterAllMatch.16k
715
715
715
—

Dict.FilterAllMatch.20k
781
782
781
—

Dict.FilterAllMatch.24k
848
849
848
—

Dict.FilterAllMatch.28k
1394
1440
1410
—

Removed

DictionaryCopy
53472
58353
55193
—

DictionaryFilter
44538
44639
44583
—

Code size: -O

TEST
OLD
NEW
DELTA
RATIO

Regression

DictionaryCopy.o
7885
10182
+29.1%
0.77x

Performance: -Osize

TEST
OLD
NEW
DELTA
RATIO

Regression

SetSubtractingInt25
102
111
+8.8%
0.92x (?)

ObjectiveCBridgeStubNSDateRefAccess
343
371
+8.2%
0.92x (?)

Improvement

DataAppendDataLargeToLarge
51000
37400
-26.7%
1.36x (?)

Added

Dict.CopyKeyValue.16k
1587
1961
1717
—

Dict.CopyKeyValue.20k
1745
1762
1754
—

Dict.CopyKeyValue.24k
1946
1967
1960
—

Dict.CopyKeyValue.28k
3050
3572
3225
—

Dict.FilterAllMatch.16k
742
745
744
—

Dict.FilterAllMatch.20k
811
812
811
—

Dict.FilterAllMatch.24k
879
887
882
—

Dict.FilterAllMatch.28k
1449
1464
1455
—

Removed

DictionaryCopy
73938
78856
75645
—

DictionaryFilter
46272
46457
46371
—

Code size: -Osize

TEST
OLD
NEW
DELTA
RATIO

Regression

DictionaryCopy.o
6177
7958
+28.8%
0.78x

Performance: -Onone

TEST
OLD
NEW
DELTA
RATIO

Improvement

ObjectiveCBridgeStubFromNSDateRef
5480
4510
-17.7%
1.22x (?)

ObjectiveCBridgeStubFromNSDate
7460
6450
-13.5%
1.16x (?)

Added

Dict.CopyKeyValue.16k
4662
5314
4881
—

Dict.CopyKeyValue.20k
5548
5683
5636
—

Dict.CopyKeyValue.24k
6502
6699
6570
—

Dict.CopyKeyValue.28k
8619
9410
8884
—

Dict.FilterAllMatch.16k
3235
3280
3252
—

Dict.FilterAllMatch.20k
3774
3816
3788
—

Dict.FilterAllMatch.24k
4376
4424
4392
—

Dict.FilterAllMatch.28k
6096
6120
6110
—

Removed

DictionaryCopy
297362
305285
300035
—

DictionaryFilter
214145
214664
214334
—

✅
Benchmark Check Report

⚠⏱
Dict.FilterAllMatch.28k execution took at least 1392 μs.  Decrease the workload of Dict.FilterAllMatch.28k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.FilterAllMatch.28k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.28k mem_pages [i1, i2]: min=[757, 757] 𝚫=0 R=[39, 39]

⚠⏱
Dict.CopyKeyValue.20k execution took at least 1105 μs.  Decrease the workload of Dict.CopyKeyValue.20k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.20k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.20k mem_pages [i1, i2]: min=[368, 368] 𝚫=0 R=[39, 40]

⚠Ⓜ
Dict.CopyKeyValue.16k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.16k mem_pages [i1, i2]: min=[369, 368] 𝚫=1 R=[39, 40]

⚠⏱
Dict.CopyKeyValue.28k execution took at least 1888 μs.  Decrease the workload of Dict.CopyKeyValue.28k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.28k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.28k mem_pages [i1, i2]: min=[756, 755] 𝚫=1 R=[39, 40]

⚠Ⓜ
Dict.FilterAllMatch.24k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.24k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[0, 39]

⚠Ⓜ
Dict.FilterAllMatch.16k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.16k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[39, 0]

⚠⏱
Dict.CopyKeyValue.24k execution took at least 1244 μs.  Decrease the workload of Dict.CopyKeyValue.24k by a factor of 2 (10), to be less than 1000 μs.

⚠Ⓜ
Dict.CopyKeyValue.24k has very wide range of memory used between independent, repeated measurements.  Dict.CopyKeyValue.24k mem_pages [i1, i2]: min=[368, 368] 𝚫=0 R=[39, 39]

⚠Ⓜ
Dict.FilterAllMatch.20k has very wide range of memory used between independent, repeated measurements.  Dict.FilterAllMatch.20k mem_pages [i1, i2]: min=[369, 369] 𝚫=0 R=[0, 39]

How to read the data
The tables contain differences in performance which are larger than 8% and
differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

@swift-ci please test

@swift-ci please test

@eeckstein @lorentey Please review 🙏

Build failed
Swift Test Linux Platform
Git Sha - e13e90ba77c38662c826d3579ef75e71e2cd9d10

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux

@swift-ci smoke test linux platform

@swift-ci smoke test linux platform

@swift-ci smoke test linux platform

@swift-ci smoke test linux platform

@eeckstein I believe this is now mergeable: the full test that runs Benchmark_O.test.md passed on mac os and the previously broken Linux smoke test (unrelated to this PR) now also passed (full test there is not required as the benchmark validation does not run on Linux).

I'll let @lorentey review this

I have also noticed the filter has been much flatter in response, but the quadratic behavior is still fully visible in the -Onone build. The filter in optimized builds had less pronounced peak at the low end since PR #19213 when you switched it to work directly with native dictionary.

Ah, that makes sense 👍

@lorentey Thank you!


Enable .svg and Adobe file extentsion uploading to spee.ch

The .svg file type is the most common vector graphic file type so it's important that we add the option to upload .svg files to spee.ch, in addition to .ai and potentially .psd and other Adobe file extensions as soon as possible.
That being said, .svg is more urgent than the Adobe extensions.
Acceptance Criteria

Definition of Done

[ ]  Tested against acceptance criteria
[ ] Tested against the assumptions of the user story
[ ] The project builds without errors
[ ] Unit tests are written and passing
[ ] Tests on devices/browsers listed in the issue have passed
[ ] QA performed & issues resolved
[ ] Refactoring completed
[ ] Any configuration or build changes documented
[ ] Documentation updated
[ ] Peer Code Review performed

Supporting publishing of .ai files is quite different from svg. .ai are application/postscript and speech is largely hardcoded for image and mp4.

[x] publishing shows a detailed thumbnail (+preferably a descriptive one)
[x] publishing validates size on both client and server and reports errors
[x] settings supplies size limit settings for application media type and/or custom file types
[ ] * a list of default file thumbnails per contentType would be great, or at least per mediaType (text, image, etc)
[ ] at least for non-image types, a thumbnail file chooser/generator, also #622, #599

@jessopb to separate out into separate issues.


Create API 500 alarm

Follow up action from incident retro

Closing because covered in https://github.com/wellcometrust/platform-api/pull/510


⚠ API Platform (ETU) has degraded performance

In 533387d, API Platform (ETU) (https://api-etu.six-group.com/healthcheck) experienced degraded performance:

HTTP code: 503
Response time: 12610 ms

Resolved: API Platform (ETU) performance has improved in 52b73d8.


IAM permissions issues

I'm getting permissions problems:
 16:08:31<EMAIL_ADDRESS>{/usr/local/bin}$  hyperctl ami
time="2021-01-19T16:08:34-05:00" level=error msg="AccessDeniedExceptionAccessDeniedException: You don't have permissions to access this resource.\n\tstatus code: 400, request id: 610b4c58-56bd-4dcd-a3ee-0207da82eff1"
time="2021-01-19T16:08:35-05:00" level=error msg="Failed to fetch AMI UnauthorizedOperation: You are not authorized to perform this operation.\n\tstatus code: 403, request id: 8364c6a5-d7ff-4dee-9248-dda6b77aba45"
time="2021-01-19T16:08:35-05:00" level=info msg="fetched latest AMI ()" ami= name=
 16:08:35<EMAIL_ADDRESS>{/usr/local/bin}$  hyperctl clusters
time="2021-01-19T16:09:06-05:00" level=error msg="failed to list clusters, ResourceNotFoundException: Requested resource not found\n\tstatus code: 400, request id: MRVDOIRPLSCQS5FVQNSLNH018NVV4KQNSO5AEMVJF66Q9ASUAAJG"
failed to get cluster list, ResourceNotFoundException: Requested resource not found
	status code: 400, request id: MRVDOIRPLSCQS5FVQNSLNH018NVV4KQNSO5AEMVJF66Q9ASUAAJG

which IAM roles do I need?


Is this project dead?  I don't see updates since 2013 to puppetforge

Check out https://github.com/yo61/puppet-logrotate


What license is this code released under?

MIT

Can you add a license file in that case, please?

Done


Make sure that organisation env KUBERNETES_VERSION is used correctly in all K8s integration repos

Description

Check all jobs in integration k8s aws/interdomain/gke/packet/aks/kind
Make sure that all single jubs (except jobs with  version permutation) are using KUBERNETES_VERSION

@edwarnicke FYI: Thanks to @NikitaSkrynnik's efforts, we're able to update the k8s version for all our integration repositories via setting one env KUBERNETES_VERSION from the NSM organization. At this moment, it's v1.28.0 (note: for kind, we still are using a set of actual K8S versions).


WIP: fix service enable idempotency in sles11

Signed-off-by: dheerajd-msys<EMAIL_ADDRESS>Description
For service resource action: enable is not idempotent in case of SLES v11 because it uses insserv for system initialization. Service runlevels in SLES resides inside rc.d directory which was missing in conditional.
With this change action :enable is idempotent so there is no deviation to enable the service for each chef-client run.
Issues Resolved
fixes https://github.com/chef/chef/issues/6870 and https://getchef.zendesk.com/agent/tickets/21373
Recipe used for test
package 'ntp'
service 'ntp' do
  action [:enable, :start]
end

In kitchen test I've used :
  multiple_converge: 2
  enforce_idempotency: true

to ensure idempotency.
Output of kitchen test after change
PS E:\Backup\Project\Chef_Repo\sap_ntp> kitchen converge
-----> Starting Kitchen (v1.24.0)
WARN: Unresolved specs during Gem::Specification.reset:
      timeliness (~> 0.3)
      term-ansicolor (>= 0)
      specinfra (~> 2.10, ~> 2.72)
      win32-taskscheduler (~> 2.0)
      win32-certstore (>= 0.1.8)
WARN: Clearing out unresolved specs.
Please report a bug if this causes problems.
-----> Converging <default-sles-11>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 7.0.7...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
       chef installation detected
       install_strategy set to 'once'
       Nothing to install
       Transferring files to <default-sles-11>
       Starting Chef Client, version 14.10.9
       resolving cookbooks for run list: ["sap_ntp::default"]
       Synchronizing Cookbooks:
         - sap_ntp (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: sap_ntp::default
         * zypper_package[ntp] action install (up to date)
         * service[ntp] action enable (up to date)
         * service[ntp] action start (up to date)

       Running handlers:
       Running handlers complete
       Chef Client finished, 0/3 resources updated in 01 seconds
       Starting Chef Client, version 14.10.9
       resolving cookbooks for run list: ["sap_ntp::default"]
       Synchronizing Cookbooks:
         - sap_ntp (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: sap_ntp::default
         * zypper_package[ntp] action install (up to date)
         * service[ntp] action enable (up to date)
         * service[ntp] action start (up to date)

       Running handlers:
         - #<Class:0x0000000000e79cf8>::UpdatedResources
       Running handlers complete
       Chef Client finished, 0/3 resources updated in 01 seconds
       Downloading files from <default-sles-11>
       Finished converging <default-sles-11> (0m35.56s).
-----> Kitchen is finished. (0m48.20s)

Check List

[ ] New functionality includes tests
[ ] All tests pass
[ ] RELEASE_NOTES.md has been updated if required (not required for bugfixes, required for API changes)
[ ] All commits have been signed-off for the Developer Certificate of Origin. See https://github.com/chef/chef/blob/master/CONTRIBUTING.md#developer-certification-of-origin-dco

@btm, @tas50 @lamont-granquist Please take a look and review. I may need some more time to complete the functional test and customer need this fix to be merged in chef 14 as @devoptimist conveyed.
Thank you.

@ddubey The kitchen output which you are given is all from required OS ? Because I didn't see any particular code change after @devoptimist mentioned ? Could you please add more details about how it won't break on CentOS/RHEL. I think we should have functional test case in place before merging.

Thank @Vasu1105 I've also verified the updated code on CentOS v6. There are 2 cases here depending on the OS.
Service init scripts are located in below directory :
1. /etc/rc.d/rc<run_level>.d/S<priority-order><service_name> for `SUSE 11`

2. /etc/rc<run_level>.d/S<priority-order><service_name>  for `RHEL/CentOS`

So updated code works on nested folders too and checks the init scripts existence.
OUTPUT on CentOS v6
PS E:\Backup\Project\Chef_Repo\sap_ntp> kitchen converge
-----> Starting Kitchen (v1.24.0)
Please report a bug if this causes problems.
-----> Converging <default-centos6>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 7.0.7...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
       chef installation detected
       install_strategy set to 'once'
       Nothing to install
       Transferring files to <default-centos6>
       Starting Chef Client, version 14.10.9
       resolving cookbooks for run list: ["sap_ntp::default"]
       Synchronizing Cookbooks:
         - sap_ntp (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: sap_ntp::default
         * yum_package[ntp] action install (up to date)
         * service[ntpd] action enable (up to date)
         * service[ntpd] action start
           - start service service[ntpd]

       Running handlers:
       Running handlers complete
       Chef Client finished, 1/3 resources updated in 02 seconds
       Starting Chef Client, version 14.10.9
       resolving cookbooks for run list: ["sap_ntp::default"]
       Synchronizing Cookbooks:
         - sap_ntp (0.1.0)
       Installing Cookbook Gems:
       Compiling Cookbooks...
       Converging 2 resources
       Recipe: sap_ntp::default
         * yum_package[ntp] action install (up to date)
         * service[ntpd] action enable (up to date)
         * service[ntpd] action start (up to date)

       Running handlers:
         - #<Class:0x0000000000e88848>::UpdatedResources
       Running handlers complete
       Chef Client finished, 0/3 resources updated in 02 seconds
       Downloading files from <default-centos6>
       Finished converging <default-centos6> (0m47.73s).
-----> Kitchen is finished. (1m2.96s)

Customer wants this fix in chef v14 as updated by @devoptimist .

I tested this branch on debian stretch and jesse and it's fine. It doesn't really matter, as they use they systemd and debian service providers though. I didn't mean to test on stretch, but jesse still has /sbin/insserv so I wanted to see what happened. The recipe I was testing was:
service "cron" do
  action :enable
end
service "cron" do
  action :disable
end
service "cron" do
  action :enable
end
service "cron" do
  action :enable
end

The change in the regex wouldn't matter either though:
 :001 > Dir.glob("/etc/rc**/**/S*rsyslog")
 => ["/etc/rc3.d/S01rsyslog", "/etc/rc2.d/S01rsyslog", "/etc/rc5.d/S01rsyslog", "/etc/rc4.d/S01rsyslog"]
 :002 > Dir.glob("/etc/rc**/S*rsyslog")
 => ["/etc/rc3.d/S01rsyslog", "/etc/rc2.d/S01rsyslog", "/etc/rc5.d/S01rsyslog", "/etc/rc4.d/S01rsyslog"]

On Debian Jesse I did test forcing the cron provider and it works as expected, i.e.:
service "cron" do
  action :enable
  provider Chef::Provider::Service::Init::Insserv
end
service "cron" do
  action :disable
  provider Chef::Provider::Service::Init::Insserv
end
service "cron" do
  action :enable
  provider Chef::Provider::Service::Init::Insserv
end
service "cron" do
  action :enable
  provider Chef::Provider::Service::Init::Insserv
end

root@ip-172-31-44-230:/home/admin/chef# chef-apply /tmp/cron.rb
Recipe: (chef-apply cookbook)::(chef-apply recipe)
  * service[cron] action enable (up to date)
  * service[cron] action disable
    - disable service service[cron]
  * service[cron] action enable
    - enable service service[cron]
  * service[cron] action enable (up to date)


WIP: Add doc page about migrating from smtpd

Based on this SO question I think we should have a doc page about migrating from smtpd to aiosmtpd where we can highlight the important differences.
Currently I have this rough draft that could possibly be more complete (with a slightly larger example maybe), possibly go more into decode_data (which is different for py34, py35, and py36 versions of smtpd), possibly need some copy-editing for tone of language.

I've incorporated your suggestions and added imports (of smtpd, asyncore, asyncio, and aiosmtpd.controller) to the code examples.

Looks great thanks!  I'll merge this as soon as CI passes.