Removing optional  from Swift snippets

What type of PR is this? (check all applicable)

[X] 📝 Documentation Update

Description
Removed nil externalId property from Swift messages
Related Tickets & Documents
Resolves #1350

To see the specific tasks where the Asana app for GitHub is being used, see below:

https://app.asana.com/0/0/1206921594576825

LGTM! But there seems to be an error with the package.resolved.

LGTM! But there seems to be an error with the package.resolved.

re-ran and it's passed now


Stories shortcode API

Create a stories shortcode API in order to embed stories within shortcode blocks and classic editor.
Usage

Insert this shortcode in either Shortcode Block or Classic Editor.
Save the Post and Visit it on Frontend.

[stories view="circles" number="2" class="my-stories-wrapper" /]

Result

Shortcode Attributes.

view: View type for the stories. Example: circles, grid, list etc.
columns: Number of columns for the stories.
title: Whether to show title in the stories.
author: Whether to show author name in the stories.
date: Whether to display date in the stories.
story_poster: Whether to show story poster.
archive_link: Whether to display stories archive link.
archive_label: Label for archive label.
list_view_image_alignment: Useful in List mode. Alignment for image in list mode.
class: Classes for the stories wrapper.
number: How many number of stories to display. Max: 100
order: Order for rendering stories. Stories are rendered based on their publish timing. Possible options are ASC/DESC

Codecov Report

Merging #113 (8372579) into main (ac28814) will increase coverage by 8.40%.
The diff coverage is 76.64%.

@@            Coverage Diff             @@
##             main     #113      +/-   ##
==========================================
+ Coverage   75.96%   84.37%   +8.40%
==========================================
  Files         984     1007      +23
  Lines       17357    17921     +564
==========================================
+ Hits        13186    15121    +1935
+ Misses       4171     2800    -1371

Flag
Coverage Δ

karmatests
73.12% <ø> (+21.03%)
:arrow_up:

unittests
65.85% <ø> (+0.01%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

includes/Model/Story.php
77.14% <0.00%> (-9.07%)
:arrow_down:

includes/Stories_Renderer/Renderer.php
70.89% <70.89%> (ø)

includes/Shortcode/Stories_Shortcode.php
75.47% <75.47%> (ø)

includes/Story_Query.php
88.23% <88.23%> (ø)

includes/Stories_Renderer/Generic_Renderer.php
95.45% <95.45%> (ø)

includes/Stories_Renderer/Carousel_Renderer.php
95.65% <95.65%> (ø)

includes/Plugin.php
100.00% <100.00%> (ø)

...components/panels/animation/directionRadioInput.js
81.81% <0.00%> (-9.10%)
:arrow_down:

...src/edit-story/components/header/buttons/update.js
91.66% <0.00%> (-8.34%)
:arrow_down:

...story/components/panels/animation/effectChooser.js
54.83% <0.00%> (-4.43%)
:arrow_down:

... and 239 more

Closing in favour of #153


ValueError: Namespace Wnck not available for version 3.0

For problems with manjaro I installed Antergos and throw the next error:
`
Traceback (most recent call last):
File "bot.py", line 9, in
from gui.main import BotWindow
File "/home/gambl3r/Descargas/Dindo-Bot-master/gui/main.py", line 7, in
from lib import tools, logger, data, parser, settings, accounts, maps
File "/home/gambl3r/Descargas/Dindo-Bot-master/lib/tools.py", line 8, in
gi.require_version('Wnck', '3.0')
File "/usr/lib/python3.7/site-packages/gi/init.py", line 131, in require_version
(namespace, version))
ValueError: Namespace Wnck not available for version 3.0
`

hum, this means that the python Wnck package is missing in antergos, & i guess the package name is python2-wnck.
So you can try installing it using: sudo pacman -S python2-wnck
Otherwise, if you want to avoid new/next errors related to missing package(s), i invite you to try ubuntu budgie 18.04 (a debian based linux distribution) or Xubuntu if you like the XFCE Desktop environment like in Manjaro.


sway-lsp: Collect storage token in sway_ast::Attribute

This PR collects the storage attribute into the TokenMap type used in the language server. In order to do so, we are now looping through the items in sway_parse so we can access tokens in sway_ast::Attribute. The storage token in an attribute is being assigned a kind of SymbolKind::BuiltinAttr which matches the semantic definition of what rust-analyzer is doing.
Also removes an old unused test_typed_parse function.
@SilentCicero this (should) solve the highlighting issue with the storage token.

Thanks for the review @Centril. I'm going to close this in favor of accessing the attribute tokens in the ParseProgram once #2792 lands.


(Fixed)[PXN-4214] - fix text style in installment header

(Fixed)PXN-4214 - fix text style in installment header
What have changed?
fixed text style in installment header

Change looks good! Can we get before/after prints? :)

Change looks good! Can we get before/after prints? :)

done

Is there a case where the text will be displayed and does not need to have the UIColor.Andes.graySolid040 background?
Does this backgorund correspond to one selected parcel?
What if we have more parcels? It's possible?


Speed up search, also add a spinner state

In my testing of the vercel site search is too slow, 1.75s currently vs about 120ms on the current site. e.g. a search for news https://flathub.vercel.app/apps/search/news
at the very least we're going to need a spinner for transitions we expect to take more than 500ms. This may be a symptom of not being on production hardware, in which case it can be resolving by #62

Resolved by #62 - which is great news.


Does this still work?

I'm on Linux (Debian flavour).  I use FUSE a lot already for encfs, google, dropbox etc.
I've been using the onedrive-cli for awhile.  Tried installing and using this.  There was no issue with setting it up, but when I try to mount the drive, it just sits there.  I've tried on two systems.  Same result.  I'll sit there for hours.
Is there a debug or logging to check on things.  onedrivecli is working, I just can't get the FUSE wrapper working.

No idea if still works, haven't used it pretty much since I wrote it (and didn't find any use for it).
I think fuse.py (which is used here to interact with fuse) has "debug" option to at least print what happens, and maybe errors too? I dunno.
And there's --debug option in onedrivefs script itself.
You'd probably also want to use some "run in the foreground" switch with those.

I know it's a bit necro, and you've probably moved on, but as I did had this notification marked, randomly decided to check on it...
And the thing still "just works" for me, surprisingly.
Furthermore, --debug switch which I suggested above seem to provide quite a lot of comprehensive info, e.g.::
fraggod@malediction:-hatch/onedrive-fuse-fs% ./onedrivefs --debug /mnt/tmp
DEBUG:root:FS options: {'api_cache_scale': 1.0, 'api_cache': True}; FUSE: {'foreground': True}
DEBUG:root:-> init / ()
DEBUG:root:<- init None
DEBUG:root:-> getattr / (None,)
DEBUG:onedrive.api_v5:Using "requests" module version: '2.6.0'
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): apis.live.net
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive?access_token=... HTTP/1.1" 401 98
DEBUG:onedrive.api_v5:Refreshing access_token
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): login.live.com
DEBUG:requests.packages.urllib3.connectionpool:"POST /oauth20_token.srf HTTP/1.1" 200 1446
DEBUG:onedrive.conf:Updating configuration file ('/home/fraggod/.lcrc')
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive?access_token=... HTTP/1.1" 200 316
DEBUG:iso8601.iso8601:Parsed 2015-03-12T09:43:35+0000 into {'tz_sign': u'+', 'second_fraction': None, 'hour': u'09', 'daydash': u'12', 'tz_hour': u'00', 'month': None, 'timezone': u'+0000', 'second': u'35', 'tz_minute': u'00', 'year': u'2015', 'separator': u'T', 'monthdash': u'03', 'day': None, 'minute': u'43'} with default timezone <iso8601.iso8601.Utc object at 0xb69293ac>
DEBUG:iso8601.iso8601:Got u'00' for 'tz_hour' with default None
DEBUG:iso8601.iso8601:Got u'00' for 'tz_minute' with default None
DEBUG:iso8601.iso8601:Got u'2015' for 'year' with default None
DEBUG:iso8601.iso8601:Got u'03' for 'monthdash' with default 1
DEBUG:iso8601.iso8601:Got 3 for 'month' with default 3
DEBUG:iso8601.iso8601:Got u'12' for 'daydash' with default 1
DEBUG:iso8601.iso8601:Got 12 for 'day' with default 12
DEBUG:iso8601.iso8601:Got u'09' for 'hour' with default None
DEBUG:iso8601.iso8601:Got u'43' for 'minute' with default None
DEBUG:iso8601.iso8601:Got u'35' for 'second' with default None
DEBUG:root:<- getattr {u'st_mtime':<PHONE_NUMBER>, u'st_size':<PHONE_NUMBER>L, u'st_nlink': 2, u'st_mode': 16877}
DEBUG:root:-> opendir / ()
DEBUG:root:<- opendir 0
DEBUG:root:-> readdir / (0L,)
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive/files?access_token=... HTTP/1.1" 200 1656
DEBUG:root:<- readdir [u'.', u'..', u'data', u'Documents', u'me', u'Pics', u'tmp', u'path%with%20percents.txt', u'README.md', u'test.txt', u'zerofile', u'\u4e2d\u6587.txt']
DEBUG:root:-> getattr /data (None,)
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive/files?access_token=...&limit=500 HTTP/1.1" 200 1671
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/folder....?access_token=... HTTP/1.1" 200 385
DEBUG:iso8601.iso8601:Parsed 2014-04-06T00:24:47+0000 into {'tz_sign': u'+', 'second_fraction': None, 'hour': u'00', 'daydash': u'06', 'tz_hour': u'00', 'month': None, 'timezone': u'+0000', 'second': u'47', 'tz_minute': u'00', 'year': u'2014', 'separator': u'T', 'monthdash': u'04', 'day': None, 'minute': u'24'} with default timezone <iso8601.iso8601.Utc object at 0xb69293ac>
DEBUG:iso8601.iso8601:Got u'00' for 'tz_hour' with default None
DEBUG:iso8601.iso8601:Got u'00' for 'tz_minute' with default None
DEBUG:iso8601.iso8601:Got u'2014' for 'year' with default None
DEBUG:iso8601.iso8601:Got u'04' for 'monthdash' with default 1
DEBUG:iso8601.iso8601:Got 4 for 'month' with default 4
DEBUG:iso8601.iso8601:Got u'06' for 'daydash' with default 1
DEBUG:iso8601.iso8601:Got 6 for 'day' with default 6
DEBUG:iso8601.iso8601:Got u'00' for 'hour' with default None
DEBUG:iso8601.iso8601:Got u'24' for 'minute' with default None
DEBUG:iso8601.iso8601:Got u'47' for 'second' with default None
DEBUG:root:<- getattr {u'st_mtime':<PHONE_NUMBER>, u'st_size':<PHONE_NUMBER>L, u'st_nlink': 2, u'st_mode': 16877}
DEBUG:root:-> getattr /Documents (None,)
...

So I wonder if you did e.g. run onedrivefs /some/path, it mounted the thing, and "just sat there" waiting for requests from FUSE layer.
And in case it's ls /some/path commands that are hanging, that --debug switch should probably help.
Didn't really expect it to work on the first try even for me, but here we go.

durdle@t420-ssd [58C] [] ~ $ ./onedrivefs --debug /mnt/disk1
DEBUG:root:FS options: {'api_cache_scale': 1.0, 'api_cache': True}; FUSE: {'foreground': True}
DEBUG:root:-> init / ()
DEBUG:root:<- init None
DEBUG:root:-> getattr /.Trash (None,)
DEBUG:onedrive.api_v5:Using "requests" module version: '2.6.0'
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): apis.live.net
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive/files?access_token=REMOVE HTTP/1.1" 401 98
DEBUG:onedrive.api_v5:Refreshing access_token
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): login.live.com
DEBUG:requests.packages.urllib3.connectionpool:"POST /oauth20_token.srf HTTP/1.1" 200 1422
DEBUG:onedrive.conf:Updating configuration file ('/home/durdle/.lcrc')
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive/files?access_token=REMOVE HTTP/1.1" 200 6707
DEBUG:root:<- getattr '[Unhandled Exception]'
DEBUG:root:-> getattr /.Trash-1000 (None,)
DEBUG:requests.packages.urllib3.connectionpool:"GET /v5.0/me/skydrive/files?access_token=REMOVE HTTP/1.1" 200 6704
DEBUG:root:<- getattr '[Unhandled Exception]'
To prove onedrive-cli works....
durdle@t420-ssd [59C] [] ~ $ onedrive-cli info
client_updated_time: 2014-10-31T13:03:45+0000
comments_count: 0
comments_enabled: False
count: 48
created_time: None
description:
from:
id: None
name: None
id: folder.REMOVE
is_embeddable: False
link: https://onedrive.live.com?cid=REMOVE
name: SkyDrive
parent_id: None
shared_with:
access: Just me
size:<PHONE_NUMBER>905
type: folder
updated_time: 2015-04-03T14:08:04+0000
upload_location: https://apis.live.net/v5.0/folder.REMOVE/files/

If I run as root...
t420-ssd durdle # onedrivefs --debug /mnt/disk1
DEBUG:root:FS options: {'api_cache_scale': 1.0, 'api_cache': True}; FUSE: {'foreground': True}
DEBUG:root:-> init / ()
DEBUG:root:<- init None

Running as root shouldn't be necessary, but above --debug output clearly shows that something does DEBUG:root:-> getattr /.Trash (None,) that ends up throwing some exception, probably due to something like http-400, as OneDrive doesn't allow names to start with "." iirc.
"something" that does that getattr is probably desktop environment (like GNOME).
Guess such errors from the API should be handled, weird that they aren't, will check.

Checked, apparently DEBUG:root:<- getattr '[Unhandled Exception]' works for me as ENOENT, so probably should work for you in the same way too.
So, again, if the issue seem to be:
There was no issue with setting it up, but when I try to mount the drive, it just sits there.

This seem to be the expected result - process just runs in the foreground until you stop it.
Did you try doing e.g. "ls" on that path from different shell?

Ah yes, it does work lol
OneDriveFS<PHONE_NUMBER>0<PHONE_NUMBER><PHONE_NUMBER>  50% /mnt/disk1
I tried doing a ls on the mount point before and it wasn't returning.  Now that I have debug on, I can see it makes a web call for each file+directory within root, so takes a while on a root with 50 files/directories.  I let it run while I watched debug.  It finally returned the list of contents.  Perfect.
This will be handy.

I think it should be way faster if it was simply opendir(2) + readdir(2) - that's just a single onedrive call, as you can probably see in traces, but on most distros and configurations, "ls" is aliased to something like ls --color=auto --group-directories-first, which it does in only way POSIX fs semantics allow:

stat on specified dir
opendir
readdir
stat on each name in that dir, sequentially
getxattr on each name in that dir, sequentially

I.e. readdir(2) there can't return attributes for each file/dir (and whether it's a file or dir at all), just a list of names, and for --group-directories-first ls needs mode for each name, and for --color=auto I think it also checks xattrs (to e.g. mark stuff with posix capabilities red).
Furthermore, old API doesn't even have anything like paths, only id's linked to each other, with "name" attribute attached to each, so "ls path1/path2/path3" will do 3 requests just to "resolve" that path (list root, find "path1" there, list path1-id, find "path2" there, etc).
Stuff can be cached, e.g. whatever readdir returns (which is later queried by stat), and that's what I started implementing with "CacheLRU" and "PerPathCache" classes, but then you run into a problem of its invalidation, it gets complicated fast.
Not anywhere near impossible, just something should be implemented by someone who cares, then I think it can work, at least as long as you have only one client messing with the remote stuff.
So I guess two points I'd like to make here:

To make it faster, do unalias ls and avoid anything that fancy.
To make it even faster, stop using OneDrive through POSIX FS interface entirely, which it does not implement (or maps to) natively, hence the inherent inefficiency/complexity/bugs of such translation.

sudo onedrivefs  --debug /mnt/onedrive
DEBUG:root:FS options: {'api_cache_scale': 1.0, 'api_cache': True}; FUSE: {'foreground': True}
DEBUG:root:-> init / ()
DEBUG:root:<- init None

onedrive-cli ls/get/put works like a charm

@pashazz
Looks like command you ran worked, as there's no error, guess it's just not obvious that it doesn't fork itself into background by default.

By the way, don't think you need "sudo" there.

Changed default behavior (as of bbd6a93) to go into background, but --debug now implies --foreground (as otherwise there will be no output), so be sure to drop that option if you want it to fork out from console.
Should do the same thing with the command you've used - i.e. go up to init and wait/serve requests to fs from there.

Thanks!

 onedrivefs ~/.lcrc /mnt/onedrive
/usr/bin/env: python22: No such file or directory

Fix it please. Of course I changed the first line for myself.

Uhhh.... it has "python2" in there - https://github.com/mk-fg/onedrive-fuse-fs/blob/master/onedrivefs#L1
Not sure what changed it to "python22" for you, maybe some patch you've applied on top?

May be. I use Arch...apparently they f**ked up the pkgbuild :D

Apparently so - https://aur.archlinux.org/packages/on/onedrive-fuse-fs-git/PKGBUILD - sed line there does that, as I've replaced python with python2 in the repo (it's the right thing to do).
I think I've been bitten by not including that "$" in regexp wrt python shebangs a few times already, seem to be an easy mistake to make. Should be easy to fix though.


Add more measurements to experiments

To showcase how Rheem has been executed, it should log more statistics to Experiments, for instance:

[ ] actually executed ExecutionOperators
[ ] how much time has been spent in each of the Executors
[ ] or the PartialExecutions in the first place
[ ] measured cardinalities

This issue was moved to rheem-ecosystem/rheem#23


The keys of hypertries are not copied/moved during assignment

The assignment operator does not copy/move  the contents of the keys of the RHS hypertrie to the LHS hypertrie. In the following example, after the hypertrie ht goes out of scope (end of if clause), the contents of its keys get deleted. const_ht preserves the depth and size of ht, but if we try to access the contents of the keys, we get a segmentation fault.
const_Hypertrie<default_bool_Hypertrie_t> const_ht{2};
std::cout << "CONST_HT DEPTH BEFORE IF CLAUSE: " << const_ht.depth() << std::endl;
// CONST_HT DEPTH BEFORE IF CLAUSE: 2
std::cout << "CONST_HT SIZE BEFORE IF CLAUSE: " << const_ht.size() << std::
// CONST_HT SIZE BEFORE IF CLAUSE: 0
if(true) {
    Hypertrie<default_bool_Hypertrie_t> ht{3};
    ht.set({1, 10, 20}, true);
    ht.set({2, 10, 20}, true);
    ht.set({3, 10, 20}, true);
    ht.set({4, 10, 20}, true);
    ht.set({5, 10, 20}, true);
    // assignment
    const_ht = const_Hypertrie(ht);
    std::cout << "HT DATA: \n" << (std::string) ht << std::endl;
    // HT DATA:
    // [ ⟨5, 10, 20⟩ → true, ⟨1, 10, 20⟩ → true, ⟨3, 10, 20⟩ → true, ⟨4, 10, 20⟩ → true, ⟨2, 10, 20⟩ → true ]
    std::cout << "CONST_HT DATA INSIDE IF CLAUSE: \n" << (std::string) const_ht << std::endl;
    // CONST_HT DATA INSIDE IF CLAUSE:
    // [ ⟨5, 10, 20⟩ → true, ⟨1, 10, 20⟩ → true, ⟨3, 10, 20⟩ → true, ⟨4, 10, 20⟩ → true, ⟨2, 10, 20⟩ → true ]
}
std::cout << "CONST_HT DEPTH AFTER IF CLAUSE: " << const_ht.depth() << std::endl;
// CONST_HT DEPTH AFTER IF CLAUSE: 3
std::cout << "CONST_HT SIZE AFTER IF CLAUSE: " << const_ht.size() << std::endl;
// CONST_HT SIZE AFTER IF CLAUSE: 5
std::cout << "CONST_HT DATA AFTER IF CLAUSE: \n" << (std::string) const_ht << std::endl; // segmentation fault
auto first_const_ht_key = const_ht.begin(); // segmentation fault

Is this the proper way to copy the data of a hypertrie to another one?

const_Hypertrie holds a non-owning reference to the internal hypertrie data-structure.
The initialization of a depth 2 hypertrie in line 1 const_Hypertrie<default_bool_Hypertrie_t> const_ht{2}; is fine because empty tensors are not backed by any data.
But after const_ht = const_Hypertrie(ht); const_ht is bound to a internal structure that is owned by ht only. When ht goes out of scope the internal structure is automatically cleaned up.
Comparable to iterators in STL, the const_ht is invalidated when the referenced ht goes out of scope.


isAvailable event is being triggered after a 30-40 second delay, in ACS Room Video Call

Package Name: @azure/communication-calling
Package Version: 1.23.1-beta.2
Operating system: MacOS
[ ] nodejs

version: 20.11.0

[ ] browser

name/version: Chrome / 125.0.6422.76

[ ] typescript

version: 5.3.3

Describe the bug
In a room Video Call, when a participant hangs-up, isAvailable and remoteParticipantsUpdated event is being emitted after a 30-40 second delay.
While we are getting isReceiving event immediately.
This is causing issue for us, because for those 30-40 seconds, that participant's video stream is showing blank for others, resulting in a bad user experience.
Expected behavior
We are expecting these events to get triggered immediately or with small delay.
If this is an expected behaviour, then please help with dealing with this scenario. Or if there is any configuration that can take care of this issue.

Hi @virtualvishwam - can you please post a ACS Call ID ( call.id property )
I assume that you have a Rooms call - and one of the participants who's sending video - hang's up
It's not expected to experience such a delay , other participants in the call should be informed almost immediately that another participant left the call.

Hi @akania - I did try again. This time the delay between isReceiving and remoteParticipantsUpdated events was around 1 minute.
Call Id -> 9d7ab88f-0a6d-4b83-8638-31ab02963694

@virtualvishwam
How is the remoteParticipantsUpdated event happens? Does the participant close the tab/app or hangups from the call normally - using call.hangup() API.
If the user ends the call gracefully, the updates should be instant. If the remote participant just closes the tab/app there is time period where we "try" to find the participant/re-connect the call etc.

@fizampou - The participant hangs-up the call using call.hangup() API

Hi @virtualvishwam, following investigation into this issue, I was able to receive an immediate isAvailable and remoteParticipantsUpdated event in a Rooms call following a remote participant leaving the call gracefully - in a newer @azure/communication-calling version than the one you specified.
Since 1.23.1-beta.2 has since been deprecated, please ensure you update to the latest released version of Azure Communication Services Web Calling SDK to get the latest features and important fixes.
I will be closing this issue but please re-open if you encounter the same experience on a newer version. Thanks!


Chapter 6: no postgres option available

In chapter 6, we are instructed to setup a Postgres database.
But currently, there is no postgres option available without using Neon.

It seems that the information is outdated and should be updated.

For me it works with Neon right now.

For me it works with Neon right now.
I understand that doing a similar procedure can work in Neon, but I was confused by the slightly different procedures for creating databases and viewing tables in Neon.
I think it is appropriate to update the info.

I encountered the same issue, but after checking the Vercel documentation-Neon partnership, I found out that Vercel now has a partnership with Neon. Moving forward, it’s recommended to use Neon Postgres for storage with Vercel.

Hey! Yes, the Vercel Postgres option is being replaced with the Neon integration option. You won't see the old option if you've recently created a Vercel account. We're updating Chapter 6 to reflect these changes, and recently merged a PR that'll allow you to query the database using a Route Handler since the query UI is not available:  https://github.com/vercel/next-learn/pull/927
We're also testing the Supabase integration with this project, so that in the future you'll be able to choose between Neon or Supabase.

We're also testing the Supabase integration with this project, so that in the future you'll be able to choose between Neon or Supabase.

As of 11/24/2024 I can confirm that the project does not work when using Supabase (fails with 404 when calling fetchRevenue() in Chapter 7
When switching to Neon it works (up to Chapter 7's fetchRevenue() call.. I haven't proceeded any farther)


ISSUE: not receiving UPS or FedEx delivery notifications in mail and packages

Describe the bug
I am seeing my USPS deliveries in mail and packages but not UPS and FedEx. I am getting delivery notifications by email form both UPS and FedEx in the same gmail account. I have never received UPS or FedEx alerts.
thank you for your help
Environment (please complete the following information):

OS: [current HA OS and Supervisor]
Type of device: [Synology/VMM]
Home Assistant version: [current]
Hassio/Docker/Core?
Component version: [current]

Do you have the sensors enabled for FedEx and UPS?

Thank you for the quick response. Yes, I initially enabled the sensors, and when I view them in developer tools, they are available and show 0. In my card, I have them enabled.

Enable debugging and post the Home Assistant log so I can see what's happening.

I enabled the logging and I have  a UPS package coming Monday. I will upload the log file then.
Thank you

Wanted to let you know today it worked as expected. Thank you


[dask] Fix prediction on DaskDMatrix with multiple meta data.

Unify the meta info handling for training and prediction.

cc @kylejn27

Codecov Report

Merging #6333 into master will decrease coverage by 0.01%.
The diff coverage is 97.87%.

@@            Coverage Diff             @@
##           master    #6333      +/-   ##
==========================================
- Coverage   80.84%   80.82%   -0.02%
==========================================
  Files          12       12
  Lines        3373     3375       +2
==========================================
+ Hits         2727     2728       +1
- Misses        646      647       +1

Impacted Files
Coverage Δ

python-package/xgboost/dask.py
81.14% <97.87%> (-0.12%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e1e972...23ed4e4. Read the comment docs.


Higher Memory Usage with model.predict in Recent TF Versions (TF 2.10, 2.11 etc)

Click to expand!
Issue Type
Bug
Source
binary

we use pip install to reproduce the issue although we use poetry in production.

Tensorflow Version
2.10 and 2.11 and maybe others after 2.8.2 all have this issue
Custom Code
No (can reproduce the issue with simple code below)
OS Platform and Distribution
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

x86_64
Mobile device
No response
Python version
3.8.15
Bazel version
No response
GCC/Compiler version
No response
CUDA/cuDNN version
No response
GPU model and memory
No response
Current Behaviour?
We use TF to serve multiple models in production and recently we tried updating from TF 2.8.2 to TF 2.10. We noticed on many of our services/workers there was a memory increase/leak with some services even crashing. We had to revert back to 2.8.2. I also see the issue in TF 2.11. With this deploy in particular the only thing we upgraded was Tensorflow. No application code changes.
We monitor the memory with DataDog and all our host graphs showed major spikes. These are mostly tf.keras models. I thought originally it was a ddtrace issue because from TF 2.9 and up model.predict started printing progress bars and I thought all those were getting collected in  DataDog traces. But I set verbose=0 on all model.predict calls and turned off ddtrace an the issue was still there.
Some screenshots from two different services

Standalone code to reproduce the issue
Run this code on a machine with 8 cores and 16GB of ram for example.
Here is also a colab with the same code as below although not sure you can replicate the memory leak issue in colab. I am replicating the issue on a standard aws ec2 instance c5.2xlarge. Running in a single ipython session for an hour or two will show the leak. You can speed it up by running 5 ipython sessions at once in 5 shells.
Minimal Dockerfile
FROM python:3.8-slim

WORKDIR /app

RUN apt-get update && apt-get install -y \
        wget \
        htop \
        vim

COPY . /app

RUN pip install tensorflow #  pip install tensorflow==2.8.2 does not have MEM leak
RUN pip install ipython
RUN pip install Pillow
EXPOSE 5000

docker build . -t tf
docker run -it tf /bin/bash

Now in the docker container
ipython

# download an image
!wget --output-document=my_image.jpg https://www.wwf.org.uk/sites/default/files/styles/social_share_image/public/2018-10/Large_WW1113482.jpg?itok=Bluh496C

import tensorflow
from tensorflow import keras
from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input
import numpy as np

model = ResNet50(weights='imagenet', include_top=False, pooling='avg')
img_path = 'my_image.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
for _ in range(100000):
    embedding = model.predict(x)

Look at the memory usage with htop with TF 2.8.2 and the memory will not grow.
Now repeat the same thing with TF 2.10, 2.11 etc (maybe even 2.9 But I never checked it)
and you will see much higher memory usage and a linear growth.
Relevant log output
TF 2.8.2
Here is the pip freeze inside the Docker ENV for TF 2.8.2 version
absl-py==1.3.0
asttokens==2.1.0
astunparse==1.6.3
backcall==0.2.0
cachetools==5.2.0
certifi==2022.9.24
charset-normalizer==2.1.1
decorator==5.1.1
executing==1.2.0
flatbuffers==22.11.23
gast==0.5.3
google-auth==2.14.1
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.50.0
h5py==3.7.0
idna==3.4
importlib-metadata==5.1.0
ipython==8.6.0
jedi==0.18.2
keras==2.8.0
Keras-Preprocessing==1.1.2
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
numpy==1.23.5
oauthlib==3.2.2
opt-einsum==3.3.0
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
prompt-toolkit==3.0.33
protobuf==3.19.6
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.13.0
requests==2.28.1
requests-oauthlib==1.3.1
rsa==4.9
six==1.16.0
stack-data==0.6.1
tensorboard==2.8.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.8.2
tensorflow-estimator==2.8.0
tensorflow-io-gcs-filesystem==0.28.0
termcolor==2.1.1
traitlets==5.5.0
typing_extensions==4.4.0
urllib3==1.26.13
wcwidth==0.2.5
Werkzeug==2.2.2
wrapt==1.14.1
zipp==3.10.0

Memory Stays at Like 1GB

Here is the log output when you import tensorflow as tf on TF 2.8.2
2022-11-25 15:44:46.912225: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-11-25 15:44:46.912258: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

TF 2.11
And here is the pip freeze for a higher version of TF 2.11
absl-py==1.3.0
asttokens==2.1.0
astunparse==1.6.3
backcall==0.2.0
cachetools==5.2.0
certifi==2022.9.24
charset-normalizer==2.1.1
decorator==5.1.1
executing==1.2.0
flatbuffers==22.11.23
gast==0.4.0
google-auth==2.14.1
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
grpcio==1.50.0
h5py==3.7.0
idna==3.4
importlib-metadata==5.1.0
ipython==8.6.0
jedi==0.18.2
keras==2.11.0
libclang==14.0.6
Markdown==3.4.1
MarkupSafe==2.1.1
matplotlib-inline==0.1.6
numpy==1.23.5
oauthlib==3.2.2
opt-einsum==3.3.0
packaging==21.3
parso==0.8.3
pexpect==4.8.0
pickleshare==0.7.5
Pillow==9.3.0
prompt-toolkit==3.0.33
protobuf==3.19.6
ptyprocess==0.7.0
pure-eval==0.2.2
pyasn1==0.4.8
pyasn1-modules==0.2.8
Pygments==2.13.0
pyparsing==3.0.9
requests==2.28.1
requests-oauthlib==1.3.1
rsa==4.9
six==1.16.0
stack-data==0.6.1
tensorboard==2.11.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
tensorflow==2.11.0
tensorflow-estimator==2.11.0
tensorflow-io-gcs-filesystem==0.28.0
termcolor==2.1.1
traitlets==5.5.0
typing_extensions==4.4.0
urllib3==1.26.13
wcwidth==0.2.5
Werkzeug==2.2.2
wrapt==1.14.1
zipp==3.10.0

The memory will keep growing to even over 15GB.
Here is a screen shot of it on its way growing over time

Here is the log output when import tensorfow on most recent version TF 2.11
2022-11-25 15:37:47.360066: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-11-25 15:37:47.498679: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-11-25 15:37:47.498726: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-11-25 15:37:48.351066: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-11-25 15:37:48.351130: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-11-25 15:37:48.351143: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

@DrChrisLevy,
I was facing a different error while trying to execute the code. Kindly find the gist of it here and provide the dependencies which helps to debug the issue. Thank you!

@tilakrayal yes of course you need to download an image first. I updated the code and added a wget for you to download the image.  I updated the description/code slightly. Also added a colab but Im not sure you can simply reproduce the mem leak in colab.

@tilakrayal I added even more details and log outputs in the simple reproducible Docker ENV for both 2.8.2 and 2.11.
Let me know if you need anything else from my end.

It looks TF2.9.1 doesn't seem to have the memory issue.

@sachinprasadhs,
I was able to reproduce the issue on tensorflow v2.9, v2.11 and nightly. Kindly find the gist of it here.

@DrChrisLevy I am an engineer from Intel and was able to reproduce this issue on TF2.10+. This commit which changes the default value of the env var TF_RUN_EAGER_OP_AS_FUNCTION to true seems to be the causing the leak. As a work around, could you please try setting the env var to false at runtime.
e.g. TF_RUN_EAGER_OP_AS_FUNCTION=false python test_script.py

@DrChrisLevy I am an engineer from Intel and was able to reproduce this issue on TF2.10+. This commit which changes the default value of the env var TF_RUN_EAGER_OP_AS_FUNCTION to true seems to be the causing the leak. As a work around, could you please try setting the env var to false at runtime. e.g. TF_RUN_EAGER_OP_AS_FUNCTION=false python test_script.py

Cc'ing @sagunb, the author of the commit.

@DrChrisLevy from the graphs I see a higher peak memory usage which then goes back down - is there ever a case where the memory usage does not go back down?

@DrChrisLevy from the graphs I see a higher peak memory usage which then goes back down - is there ever a case where the memory usage does not go back down?

Those graphs showing the memory going back down were either services crashing or we deployed a revert to go back to TF 2.8.2. That was in our prod system just to give some more "proof" of the issue. From now on, ignore those graphs. In every case I have seen the memory does not go back down. Look at the reproducible example I provided. The memory will grow for ever.

@DrChrisLevy I am an engineer from Intel and was able to reproduce this issue on TF2.10+. This commit which changes the default value of the env var TF_RUN_EAGER_OP_AS_FUNCTION to true seems to be the causing the leak. As a work around, could you please try setting the env var to false at runtime. e.g. TF_RUN_EAGER_OP_AS_FUNCTION=false python test_script.py

I tried changing this ENV var in the simple reproducible example I provided.
vim .env.docker  and add one line to this file ---> TF_RUN_EAGER_OP_AS_FUNCTION=false
Start the docker: docker run --env-file .env.docker -it tf /bin/bash
ipython

import os
assert os.getenv('TF_RUN_EAGER_OP_AS_FUNCTION') == 'false'

This means that in the suspected commit, os.getenv("TF_RUN_EAGER_OP_AS_FUNCTION", "1") == "1" will return False.
Then copy/paste the code I provided for producing the memory leak.
Let it run for an hour or so.
I still see the memory leak occurring.
Let me know any next steps. I appreciate your help in looking into this.

@DrChrisLevy Can you please share the TF-version you tried?
I used TF-2.10 and did not see the issue. However, I see the issue in TF-2.11 again and on further investigation found that the env var was removed from TF-2.11+ with this commit
Can you please test TF-2.10?
cc: @sagunb

@DrChrisLevy Can you please share the TF-version you tried? I used TF-2.10 and did not see the issue with the env var set to false. However, I see the issue in TF-2.11 again and on further investigation found that the env var was removed from TF-2.11+ with this commit Can you please test TF-2.10?
cc: @sagunb

Good Catch. Yes I can confirm that. I updated my prev comment. Memory leak still in TF 2.11 and not in TF 2.10 if we set TF_RUN_EAGER_OP_AS_FUNCTION=false

So what would you advise on next steps? I guess one option is to run TF 2.10 with TF_RUN_EAGER_OP_AS_FUNCTION=false in our prod env. Another option would be to wait until this is sorted out in TF 2.11 or next version?

@DrChrisLevy - I have reproduced the issue and looking into how best to address it.  TF 2.9 does not have this issue.  It's only in TF 2.10 and believe it got introduced in this commit c1286fe88e1.
One workaround in TF 2.10+ is, convert the input array to a TF Dataset for example -
for _ in range(100000):
    x_ds = tf.data.Dataset.from_tensor_slices(x).batch(32) # batch size here is arbitrary
    embedding = model.predict(x_ds)

This will stop the memory leak.  It has so far only been observed when you pass arrays directly into model.predict or to other model APIs (not verified).

@sampathweb . Thanks for looking into this. We call model.predict all over our code base in lots of different places with numpy array input and various types of models. Some of the inputs expect multiple types of input etc. I don't think I will be converting everything to  tf.data.Dataset.from_tensor_slices. It's a bit of a  hack solution.
Do you have a rough estimate of how long it would take for this problem to get solved? Are we talking weeks/months?

@sampathweb, @sagunb, @sachinprasadhs , any updates on this?

@sampathweb, @sagunb, @sachinprasadhs - We are seeing this issue for TF 2.11 and we also use poetry in production. Is there a TAT on this issue as this looks like a major issue with the newer TF versions.

Update: @jsmeredith and @poulsbo from TF apis are currently assigned to take a look. Alan/Jeremy can you provide an update?

Can you provide an update on the issue? It affects our production system as well.

Its still being looked into.  Does the workaround suggested here - https://github.com/tensorflow/tensorflow/issues/58676#issuecomment-1341147862 work for you until we have an resolution?

Its still being looked into. Does the workaround suggested here - #58676 (comment) work for you until we have an resolution?

Nope. The types of dataset we use are really complex and use sophisticated data on time-series. I know how cool TF Dataset is but moving everything to TF Datset is not doable atm.

Its still being looked into.  Does the workaround suggested here - https://github.com/tensorflow/tensorflow/issues/58676#issuecomment-1341147862 work for you until we have an resolution?

It would be a pretty big refactor to do this. Also correct me if I'm wrong but aren't TF datasets more for setting up efficient pipelines to utilize CPU and GPU when training models etc. Here we are just talking about inference and serving models.

Thanks for the feedback.  We will followup and try to find the root of the memory leak to this issue.

Hey @sampathweb , just checking for another update.
Is it still being worked on?
Thank you.

Hi @DrChrisLevy, I'm taking a look at this and other memory leaks. We're actively looking into this to find the root cause.

Hello! Is there any update on fixing this memory leak issue? This leak prevents us from upgrading of tensorflow since version 2.8.0 -- quite problematic..
Thank you very much!

We're experiencing the same issue as well.
We're using TF 2.12 and I tried applying the proposed workaround mentioned in this comment to supply model.predict with tf.data.Dataset instead of np.ndarray as we were using and I can confirm that we still observe the memory leak issue.
Is there any recent development in this topic? If it affects both supplying nd.arrays and tf.data.Dataset then I presume it's affecting a lot of deployment scenarios.

Hi!
We've also faced the same issue where we were using TF 2.8 and bumped it to 2.12 and experience this memory leak.
We've applied the workaround mentioned in this comment to supply model.predict with tf.data.Dataset and I can confirm that this has fixed the memory leak issue.

Is the problem fixed in release 2.13?

What is the latest on this issue? Is there a workaround in TF 2.11? We are also seeing increased memory usage for models on TF 2.11 compared to TF 2.4

@grasskin @sampathweb
Any update on this ?

This is work in progress, hope to have an update soon.

@poulsbo @grasskin Hello! Is there any update on fixing of this memory leak problem? Thank you!

This affects many users post-2.10, and TensorFlow should prioritize fixing it. As far as can tell this is a TF runtime issue.

I'm willing to test this out and see if the original issue I raised at the very top is fixed with calling model.predict.
What version of Tensorflow do I use to test it out @sachinprasadhs ?

I'm willing to test this out and see if the original issue I raised at the very top is fixed with calling model.predict. What version of Tensorflow do I use to test it out @sachinprasadhs ?

You can use nightly version to test it out.

Hello,
I am using tensorflow==2.14.0 and numpy==1.26.0 in a virtual environment. I am using the following code to get a prediction from a tensorflow model
pred = model(images, training=False).numpy()
I tried the followings but memory leak continues to happen. The models are also working in a production k8s environment. After 20 hours roughly, pods are required to be restarted. I wrote a simple for loop and run the followings locally and measured ram usage thanks to memray library.
Ram usage increases 170 megabytes through passing 2 tensorflow models after 2000 iterations.
def start_consuming(model_mapper: ModelMapper) -> None:
    for _ in range(2000):
        image = Image.open("file_name.png")
        msg_dict = {"realtyFileId": 123, "realtyId": 234,"imageUrl": "a"}
        process_image(image=image, msg_dict=msg_dict, model_mapper=model_mapper)
        image.close()

Didn't solve the problem.

pred = model(images, training=False).numpy()
tf.keras.backend.clear_session()

Didn't solve the problem.

pred = model(images, training=False).numpy()
tf.keras.backend.clear_session()
gc.collect()

Didn't solve the problem.

images = tf.convert_to_tensor(images)
pred = model(images, training=False).numpy()

TF_RUN_EAGER_OP_AS_FUNCTION didn't solve the problem.

TF_RUN_EAGER_OP_AS_FUNCTION=false memray python main.py

@MuhammedBuyukkinaci , Im pretty sure the proposed fix has not made its way into an official TF version yet. I think we have to try tf-nightly to test it out first. I plan to test this week tf-nightly version. I think its going to make it into TF 2.15
https://github.com/tensorflow/tensorflow/pull/62176

I can confirm that the memory issue is fixed in tf-nightly:
FROM python:3.9-slim

WORKDIR /app

RUN apt-get update && apt-get install -y \
        wget \
        htop \
        vim

COPY . /app

RUN pip install tf-nightly
RUN pip install ipython
RUN pip install Pillow
EXPOSE 5000

docker build . -t tf
docker run -it tf /bin/bash

ipython

# download an image
!wget --output-document=my_image.jpg https://www.wwf.org.uk/sites/default/files/styles/social_share_image/public/2018-10/Large_WW1113482.jpg?itok=Bluh496C
import tensorflow
from tensorflow import keras
from tensorflow.keras.applications.resnet50 import ResNet50
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input
import numpy as np

model = ResNet50(weights='imagenet', include_top=False, pooling='avg')
img_path = 'my_image.jpg'
img = image.load_img(img_path, target_size=(224, 224))
x = image.img_to_array(img)
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
for _ in range(100000):
    embedding = model.predict(x)

I guess we just wait for TF 2.15 release now.

@maksym33 I am an engineer from Intel who fixed this issue in TF. I ran the reproducer included in the description above with TF 2.15 and I do not see the memory leak (or import errors) anymore. Would you be able to provide more details (ex. reproducer) about your issue? Thanks.

"bhavani-subramanian
I think the problem is related to the Python version. I did some tests on my projects. TF 2.15 python 3.10.X - no problems with memory leaks. TF 2.15 python 3.11.X - there is a memory leak.
Maybe someone can check and confirm my findings.

@Expert73
Can you provide some reproducible examples?

@DrChrisLevy
I tested your code. Everything works without memory leak on python 3.10, 3.11.
My test model is based on layers:  Dense, LSTM, Embedding. My model leaks memory on every iteration in Python 3.11.
@DrChrisLevy
@bhavani-subramanian
Where could the problem be?
Ubuntu 22.04, python 3.11 build from source code, TF 2.15 build from source code with sse 2 optimization.

@Expert73 Without having the full code it's hard to understand and reproduce the issue.
Are you sure the memory leak is in the model.predict() part of the code?

@Doch88
Yes, I tested - there is a memory leak at the predict step. I excluded this step, substituted the result as a constant - there was no memory leak. Unfortunately, I can’t publish the full code.

@Expert73 Have you tried using tcmalloc?
Basically you install libtcmalloc, an alternative memory allocator from Google with:
sudo apt-get install libtcmalloc-minimal4
Followed by sudo ldconfig to update the links.
Then, instead of running your script with:
python script.py
you run it specifying to change the memory allocator with tcmalloc:
LD_PRELOAD="$(ldconfig -p | grep libtcmalloc_minimal.so.4 | tr ' ' '\n' | grep /)" python script.py
This helped me solve some memory issues during training and with Tensorflow Serving, I hope it can also help you.

@Doch88
Thank You!
Very cool solution. It worked great in tests. I'll see how it works in production.

@Expert73 let me know if it also works in production!

I apologize for commenting on an issue that is closed and inactive for long time but this is the only relevant thread I came across when searching for similar problem I am running into when using TensorFlow-2.15. I am not even calling predict(), just fit() when training hundreds of models causes the leak.
Background
Our project consists of training behaviour models for multiple entities. One of the model is AutoEncoder based. There are thousands of entities and we have a Python script that fetches the data for each entity, trains the relevant models (inc the autoencoder) and saves them to disk. The development was on an ancient PC running custom Tensorflow wheel (as CPU does not support the newer instructions) for 2.7.0. It had just 4GB RAM but it could train more than 3000 models without any appreciable change in RAM usage. When we deployed the program in production, the hardware was newer so we could use Tensorflow-2.15.1 (from pip), but the memory usage (machine had 8GB RAM) ballooned for the program from 10% of RAM to 30% in about an hour (After training just 200 models).
Looking at Github I found this fixed issue (which I think is merged in 2.15.1), so decided to comment on this one
Reproduction
The below script can be used to simulate part of what we are doing
# Program to check for memory leak in multi model training when using TF/Keras
import argparse
import gc
import random
import sys

import numpy as np
import pandas as pd
import psutil

import tensorflow as tf
from tensorflow.keras import (layers,
                              Model)

class NestedAutoencoder(Model):
    def __init__(self, shape):
        super().__init__()
        self.shape = shape
        self.encoder = tf.keras.Sequential([
            layers.Flatten(),
            layers.Dense(self.shape[0] // 4, activation='relu'),
            layers.Dense(1, activation='relu')])
        self.decoder = tf.keras.Sequential([
            layers.Dense(self.shape[0] // 4, activation='relu'),
            layers.Dense(tf.math.reduce_prod(shape), activation='sigmoid'),
            layers.Reshape(shape)])

    def call(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

def gen_random_df(nr, nc):
    return pd.DataFrame(np.random.random(size=(nr, nc)), columns=range(nc))

def check_loss(loss, prev_loss, tolerance):
    """
    Function that checks if loss is moving towards zero and if so
    is it doing so more than tolerance
    """
    breakme = False
    if prev_loss is not None:
        if loss < 0:
            if loss < prev_loss or loss < (1 - tolerance) * prev_loss:
                breakme = True
        else:
            if loss >= prev_loss or loss > (1 - tolerance) * prev_loss:
                breakme = True
    return breakme

def gen_train_model(model_num, num_cols):
    num_rows = random.randint(100, 100000)
    df = gen_random_df(num_rows, num_cols)
    print("Model {0} - DF Shape {1}".format(model_num, df.shape))
    shape = df.shape[1:]
    model = NestedAutoencoder(shape)
    model.compile(optimizer="adam", loss="binary_crossentropy")
    # Now we train it till we don't get convergence of error
    prev_loss = None
    while True:
        hist = model.fit(
            df, df, batch_size=32, verbose=0)
        loss = np.round(hist.history['loss'][0], 4)
        breakme_ = check_loss(loss, prev_loss, 0.05)
        if breakme_:
            break
        prev_loss = loss
    return model

def get_proc_mem():
    process = psutil.Process()
    mem = process.memory_info()
    return (mem.rss / (1024.0 * 1024.0), mem.vms / (1024.0 * 1024.0), )

def cleanup():
    tf.keras.backend.clear_session()
    gc.collect()

def parse_args():
    parser = argparse.ArgumentParser(description="Memory Leak test")
    parser.add_argument("--cleanup", action="store_true", help="Perform cleanup after evry iteration")
    parser.add_argument("output_file", help="File to which iteration wise RSS and VMS data will be written in CSV")
    ns = parser.parse_args()
    return vars(ns)

def main():
    ns = parse_args()
    ver_info = sys.version_info
    ver = ".".join(str(x) for x in ver_info)
    msg = "Memory Leak test for tensorflow v{0} on Python {1}".format(tf.version.VERSION, ver)
    if ns["cleanup"]:
        msg += " with cleanup"
    else:
        msg += " without cleanup"
    print(msg)
    num_models = 100
    num_cols = 34
    mems = []
    with open(ns["output_file"], "w") as fp:
        fp.write("iter,rss_mb,vms_mb\n")
        for i in range(num_models):
            rss, vms = get_proc_mem()
            print("Before Model {0} - Memory Usage - RSS: {1} MB, VMS: {2} MB".format(i + 1, rss, vms))
            write_tuple = (i+1, rss, vms, )
            fp.write("{0}\n".format(",".join(str(x) for x in write_tuple)))
            mobj = gen_train_model(i+1, num_cols)
            if cleanup:
                del mobj
                cleanup()

if __name__ == "__main__":
    main()

Create a virtual env and install the following in that
pip install tensorflow==VERSION psutil numpy pandas

NOTE: Code does not work with >=2.16 because keras3 causes an error in decoder layer tf.math.reduce_prod(shape)
Run the script, pass --cleanup to call cleanup routines, otherwise just give filename where csv data will be written
p39_2/bin/python test_mem_leak.py --cleanup p39_tf215_clean.csv 2> /dev/null

It writes textual representation to stdout while CSV is written to the named file
Results
All testing done on a VM with 4 cores and 8GB RAM running Rocky Linux 9.3 with Python installed using dnf from official repo mirrors
Tensorflow-2.7.4 on Python3.9
Installed from pip directly (no custom wheels)
There is no appreciable memory increase. Memory may increase for 4-5 models but it falls back to same general usage
Memory Leak test for tensorflow v2.7.4 on Python 3.9.18.final.0 without cleanup
Before Model 1 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 1212.1640625 MB
Model 1 - DF Shape (26956, 34)
Before Model 2 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2009.6328125 MB
Model 2 - DF Shape (10841, 34)
Before Model 3 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2012.23046875 MB
Model 3 - DF Shape (25314, 34)
Before Model 4 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2016.56640625 MB
Model 4 - DF Shape (52203, 34)
Before Model 5 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2016.56640625 MB
Model 5 - DF Shape (12795, 34)
Before Model 6 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2026.01171875 MB
Model 6 - DF Shape (81141, 34)
Before Model 7 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2026.01171875 MB
Model 7 - DF Shape (9530, 34)
Before Model 8 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2026.01171875 MB
Model 8 - DF Shape (40791, 34)
Before Model 9 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2009.4140625 MB
....
Before Model 96 - Memory Usage - RSS: 506.4375 MB, VMS: 2133.4765625 MB
Model 96 - DF Shape (86474, 34)
Before Model 97 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2009.9140625 MB
Model 97 - DF Shape (19520, 34)
Before Model 98 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2035.3828125 MB
Model 98 - DF Shape (15344, 34)
Before Model 99 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2035.3828125 MB
Model 99 - DF Shape (96665, 34)
Before Model 100 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2147.23046875 MB
Model 100 - DF Shape (93941, 34)

Tensorflow 2.15.1 on Python3.9
Memory usage keeps increasing. There is no appreciable difference between using --cleanup or not
Memory Leak test for tensorflow v2.15.1 on Python 3.9.18.final.0 without cleanup
Before Model 1 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 1150.26953125 MB
Model 1 - DF Shape (25924, 34)
Before Model 2 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2020.5703125 MB
Model 2 - DF Shape (18662, 34)
Before Model 3 - Memory Usage - RSS: 488.421875 MB, VMS: 2030.15234375 MB
Model 3 - DF Shape (12714, 34)
Before Model 4 - Memory Usage - RSS: 491.546875 MB, VMS: 2033.33203125 MB
Model 4 - DF Shape (76301, 34)
..
Before Model 96 - Memory Usage - RSS: 1078.421875 MB, VMS: 2610.30859375 MB
Model 96 - DF Shape (96070, 34)
Before Model 97 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2544.12109375 MB
Model 97 - DF Shape (32164, 34)
Before Model 98 - Memory Usage - RSS: 985.5625 MB, VMS: 2520.32421875 MB
Model 98 - DF Shape (21681, 34)
Before Model 99 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2521.57421875 MB
Model 99 - DF Shape (21296, 34)
Before Model 100 - Memory Usage - RSS: 978.921875 MB, VMS: 2522.82421875 MB
Model 100 - DF Shape (27779, 34)

Tensorflow 2.15.1 on Python 3.11
Similar to Python3.9 there is increase in memory usage. Cleanup makes no difference
Memory Leak test for tensorflow v2.15.1 on Python 3.11.5.final.0 without cleanup
Before Model 1 - Memory Usage - RSS: 450.6875 MB, VMS: 1165.05859375 MB
Model 1 - DF Shape (80210, 34)
Before Model 2 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2037.31640625 MB
Model 2 - DF Shape (41999, 34)
Before Model 3 - Memory Usage - RSS: 508.484375 MB, VMS: 2051.47265625 MB
Model 3 - DF Shape (24103, 34)
Before Model 4 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2077.4453125 MB
Model 4 - DF Shape (16055, 34)
Before Model 5 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2053.30859375 MB
Model 5 - DF Shape (90675, 34)
Before Model 6 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2059.4921875 MB
Model 6 - DF Shape (75456, 34)
Before Model 7 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2160.39453125 MB
Model 7 - DF Shape (35685, 34)
Before Model 8 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2120.8203125 MB
...
Before Model 95 - Memory Usage - RSS: 984.8125 MB, VMS: 2540.734375 MB
Model 95 - DF Shape (29686, 34)
Before Model 96 - Memory Usage - RSS: 1011.91015625 MB, VMS: 2557.13671875 MB
Model 96 - DF Shape (34121, 34)
Before Model 97 - Memory Usage - RSS: 1020.4921875 MB, VMS: 2578.13671875 MB
Model 97 - DF Shape (47554, 34)
Before Model 98 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2556.9453125 MB
Model 98 - DF Shape (46791, 34)
Before Model 99 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2546.734375 MB
Model 99 - DF Shape (27796, 34)
Before Model 100 - Memory Usage - RSS: 1026.59765625 MB, VMS: 2569.36328125 MB
Model 100 - DF Shape (45767, 34)

So this still isn't fixed huh

Could you please try in the latest version, TensorFlow now supports Keras 3 in it's backend from TensorFlow 2.16 version.

@sachinprasadhs I have retried it with TensorFlow-2.17
To get the reproduction script (given in above comment) to work with Keras3, the autoencoder needs to be tweaked.
class NestedAutoencoder(Model):
    def __init__(self, shape):
        super().__init__()
        self.shape = shape
        self.encoder = tf.keras.Sequential([
            layers.Flatten(),
            layers.Dense(self.shape[0] // 4, activation='relu'),
            layers.Dense(1, activation='relu')])
        self.decoder = tf.keras.Sequential([
            layers.Dense(self.shape[0] // 4, activation='relu'),
            layers.Dense(tf.math.reduce_prod(shape).numpy(), activation='sigmoid'),
            layers.Reshape(shape)])

    def call(self, x):
        encoded = self.encoder(x)
        decoded = self.decoder(encoded)
        return decoded

The memory leak still persists and appears to be worse when compared to 2.15
Memory Leak test for tensorflow v2.17.0 on Python 3.11.5.final.0 without cleanup
Before Model 1 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 1298.80859375 MB
Model 1 - DF Shape (84735, 34)
Before Model 2 - Memory Usage - RSS: 511.765625 MB, VMS: 2245.68359375 MB
Model 2 - DF Shape (33172, 34)
Before Model 3 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2290.6875 MB
Model 3 - DF Shape (7670, 34)
Before Model 4 - Memory Usage - RSS:<PHONE_NUMBER> MB, VMS: 2278.5859375 MB
Model 4 - DF Shape (8959, 34)
Before Model 5 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2280.58203125 MB
Model 5 - DF Shape (81479, 34)
Before Model 6 - Memory Usage - RSS: 545.65625 MB, VMS: 2282.70703125 MB
Model 6 - DF Shape (23503, 34)
Before Model 7 - Memory Usage - RSS: 570.53125 MB, VMS: 2303.5625 MB
Model 7 - DF Shape (32387, 34)
Before Model 8 - Memory Usage - RSS: 590.46875 MB, VMS: 2323.92578125 MB
Model 8 - DF Shape (93256, 34)
Before Model 9 - Memory Usage - RSS: 579.703125 MB, VMS: 2319.53515625 MB
Model 9 - DF Shape (89587, 34)
Before Model 10 - Memory Usage - RSS:<PHONE_NUMBER>5 MB, VMS: 2391.12890625 MB
...
Before Model 90 - Memory Usage - RSS: 1225.30859375 MB, VMS: 2954.484375 MB
Model 90 - DF Shape (40989, 34)
Before Model 91 - Memory Usage - RSS: 1224.7265625 MB, VMS: 2955.43359375 MB
Model 91 - DF Shape (3085, 34)
Before Model 92 - Memory Usage - RSS: 1190.15625 MB, VMS: 2952.1171875 MB
Model 92 - DF Shape (52466, 34)
Before Model 93 - Memory Usage - RSS: 1246.5078125 MB, VMS: 2978.23828125 MB
Model 93 - DF Shape (99352, 34)
Before Model 94 - Memory Usage - RSS: 1284.52734375 MB, VMS: 3024.25390625 MB
Model 94 - DF Shape (41397, 34)
Before Model 95 - Memory Usage - RSS: 1254.96875 MB, VMS: 2987.54296875 MB
Model 95 - DF Shape (12280, 34)
Before Model 96 - Memory Usage - RSS: 1231.5 MB, VMS: 2962.69921875 MB
Model 96 - DF Shape (82651, 34)
Before Model 97 - Memory Usage - RSS: 1250.2578125 MB, VMS: 2986.26171875 MB
Model 97 - DF Shape (84752, 34)
Before Model 98 - Memory Usage - RSS: 1258.265625 MB, VMS: 2988.80859375 MB
Model 98 - DF Shape (41632, 34)
Before Model 99 - Memory Usage - RSS: 1284.04296875 MB, VMS: 3011.89453125 MB
Model 99 - DF Shape (8033, 34)
Before Model 100 - Memory Usage - RSS: 1256.125 MB, VMS: 2984.359375 MB
Model 100 - DF Shape (50343, 34)


Unreachable chunk

I tried to understand what this chunk of code should do, but I couldn't. However this is unreachable code.

This project has some code lines from decompiled version of internal google support library.


Plug-in system for URL schemes

Some app developers would like to handle networking themselves via a plugin.  This plugin system would allow an arbitrary callback to be installed to handle app-specific URL schemes.

We landed the new NetworkingEngine with this functionality in ab127e6.


Build failed

I tried several times to build gradle file. But always it shows below message.
Use JsonReader.setLenient(true) to accept malformed JSON at line 1 column 10 path $

Could you revert your changes to the dependencies to see if that helps?

Okay. I will try and let you know.
Thanks for the quick answer.

I tried your suggestion. But still, it's here. Still, I'm facing same error.


Default Dashboard: Facebook Ads "Inline Link Clicks" over Time

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/1728
Originally created by @dmor on 2020-02-13 20:28:55

Definition of "Inline Link Clicks":

The number of clicks on links to select destinations or experiences, on or off Facebook-owned properties. Inline link clicks use a fixed 1-day-click attribution window.

FYI: in the ads interface they don't include the word "inline"
For the MVC, we should just show this at the account level (no special selector for specific campaigns)
What this looks like in the Facebook UI:

Facebook Docs:

https://developers.facebook.com/docs/marketing-api/insights/parameters/v6.0

View 13 previous comments from the original issue on GitLab


gui: add new attribute filtering for selection

When converted into a PR this should be linked with #3768.
I am creating this draft to show my approach for adding the new filter: -filter attribute=value. I have neither updated the readme nor formatted properly.
The idea is that each selected object type (e. g. Instance, Net, etc.) have its own method for filtering.
Each type will have its own filtering attributes which will have its own specific values.
In the singleton select method we check the type and call a private method which handles the filtering logic. I thought that doing that can keep the nested conditions more concise.
I don't like all the string comparisons, but I don't see a different way of doing it.

@AcKoucher Would it be an idea to inject the command that does the selection into the Tcl console so the user can tinker with more advanced options? That way we wouldn't have to clutter the GUI for more obtuse options and there is something "market" the more advanced feature. The user would start with a simple selection, do up-arrow in the console and then add some more obscure options for which there are no GUI buttons. Just a tought...

@oharboe I think that injecting the selection command in the Tcl console can make sense - especially if we're trying to keep the same line of #3739 - if triggered by right-clicking the selectable cell in the inspector, although the layout viewer's selection ability is already very powerful by itself and the selection commands are shorter when compared to sta long commands. @maliberty what do you think?

One thing I like about injecting GUI operations as Tcl commands into the command line is that it will be easier to write github issues and provide instructions :-) "Paste X into the command console". Though... it is tougher to go from command line to illustrate where in the GUI you can do the same....

Some considerations after the last changes:
I think it would be nice to review the properties' syntax (I'd be happy to do that). There are some properties which are written like: Tech Layer, but in other objects it's written as Tech layer (with lower case 'l').
It would be cool to have an option in the TCL Input context menu for inserting special characters (Ω, μ, etc.). It could be use to enter something like:
select -type -filter {Resistance=75.32 mΩ/sq}

I really like this - its very powerful.


clarify wording

Greater clarity needed to distinguish between the 3rd party provider and Google services.

I signed it.

Thanks for the contribution!


Smarter imports as code action

Is your feature request related to a problem? Please describe.
Currently, if a symbol is not in scope, pyright offers two code actions for importing it:

Directly importing the symbol as a module, so a missing symbol time will cause import time to be added
Importing the symbol from a module, prompting for the module name. This requires the file to be saved and for rope to be installed.

This is cumbersome, as you need to type out the module name if the missing symbol is in a module. Instead, you can retrigger suggestions on the missing symbol and choose it from the list. The symbol will automatically be imported. The file does not need to be saved, and rope does not need to be available (as far as I'm aware).
Describe the solution you'd like
It would be nice if a code action was offered for each suggestion that matches the symbol exactly. In most cases, this should not be more than one or two.
This is mainly an issue for short symbols, where I tend to type it out fully without choosing a suggestion. My editor also requires me to manually execute the action associated with the suggestion, so commonly I just forget. (I could probably change that, but I actually prefer my code not be edited without direct action from me)
Apart from Python, I also commonly work with Rust. Rust-analyzer offers a vast array of code actions that are very helpful. Pyright seems to barely offer any, but this one in particular I miss the most, even if an alternative solution exists.
PS:
I notice the README does not actually define pyright as a language server, but merely as a type checker. If coc-pyright implements its own language server on top of pyright, then I apologize for the noise. So far I was under the impression that pyright is a full language server, but now I'm having doubts...

this functionality is already in pylance. please, try out https://marketplace.visualstudio.com/items?itemName=ms-python.vscode-pylance


feat: add environment param to events

Adds the ability to configure an environment param when initializing the client, or take it automatically from the NEW_RELIC_ENVIRONMENT environment variable, that is normally configured when using the Python agent.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Hey @tomperi, we just launched a new version that includes many features including an dedicated entity for each LLM application. I encourage you to try it, and I would be happy to hear your feedback.


Deleted almost all data models

Tchen, you've just have deleted almost all data models that we created for the project(except News)

its ok now
@bogosla


get-dbasdtc

PS C:\dbatools> Get-DbaMsdtc -ComputerName xyz
Method invocation failed because [Sqlcollaborative.Dbatools.Parameter.DbaInstanceParameter] does not contain a method named 'split'.
At C:\dbatools\allcommands.ps1:38730 char:57

    $ComputerName = $ComputerName | ForEach-Object {$_.split("\")[0]} | Sele ...

                                                    ~~~~~~~~~~~~~~~~

CategoryInfo          : InvalidOperation: (:) [], RuntimeException
FullyQualifiedErrorId : MethodNotFound

Thanks for the report! I'll take a look at this today.

Thanks...but I got the latest and it it fixed now...
Sorry for the trouble.
Cheers
Ranga
Get Outlook for Androidhttps://aka.ms/ghei36

From: Chrissy LeMaire<EMAIL_ADDRESS>Sent: Saturday, December 15, 2018 6:51:02 AM
To: sqlcollaborative/dbatools
Cc: RangaSQL; Author
Subject: Re: [sqlcollaborative/dbatools] get-dbasdtc (#4822)
Thanks for the report! I'll take a look at this today.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/sqlcollaborative/dbatools/issues/4822#issuecomment-447562798, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AbnvmPKjV2Rv2l7v2f8ZERgHYg0ecTqHks5u5OImgaJpZM4ZUMIz.

excellent, glad it's fixed!


fixed trunk: wrong way to require a module

This should fix https://travis-ci.org/facebook/react-native/jobs/149264391

@facebook-github-bot shipit


POST /cleaner/set_control_info return 404

Hi, I have one Daikin Air Purifier (MCX70XY) and I wanted to implement a plugin for Homebridge.
But I am unable to call cleaner/set_control_info API.
Can I know how you found out all the available API?

hi, dylannlaw
Control method of the official control application has changed from a certain version.
So, it seems to be duplicate in the latest firmware version.
I checked the communication of the official app by mitmproxy, it was different.

Thank you.


runTestAndWait() often causes 429 Too Many Requests errors

I noticed that runTestAndWait() has recently started to throw 429 errors. It seems to be caused by the polling the function performs.
Related: I tried to set a high number for the pollResults option (we used the default, which shouldn't cause 429 either imo :)), but it doesn't seem to do anything, the status updates (e.g., Waiting at the front of the queue...) are printed at the same-ish interval regardless of what I configure.
wpt.runTestAndWait( url, {
  location: 'gce-europe-west4',
  connectivity: 'Cable',
  runs: 1,
  firstViewOnly: true,
  keepOriginalUserAgent: true,
  pollResults: 30,
  timeout: 120,
  lighthouse: false,
})

Testing is in progress, please be patient
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Waiting at the front of the queue...
Test Started 5 seconds ago
Test Started 19 seconds ago
Error
    at Object.<anonymous> (/home/ubuntu/git/bsl-sites/bsl-speedtestmonitor/node_modules/webpagetest/lib/helper.js:213:25)
    at Module._compile (node:internal/modules/cjs/loader:1469:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
    at Module.load (node:internal/modules/cjs/loader:1288:32)
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at Module.require (node:internal/modules/cjs/loader:1311:19)
    at require (node:internal/modules/helpers:179:18)
    at Object.<anonymous> (/home/ubuntu/git/bsl-sites/bsl-speedtestmonitor/node_modules/webpagetest/lib/webpagetest.js:14:12)
    at Module._compile (node:internal/modules/cjs/loader:1469:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10) {
  name: 'WPTAPIError',
  code: 429,
  message: 'Too Many Requests'
}

Hello there,
This looks more like a support issue. I would suggest logging a ticket with WebPageTest support at<EMAIL_ADDRESS>When submitting your ticket, please include the following information:

IP Address
User Agent
Time when the issue occurred
Endpoint called

Additionally, please confirm if you are passing the key in the headers.

@AatibMeer I missed your reply. I've just reached out to support. Thanks!


Feature request: Export SVG / XML

Would be nice if you could export the generated link graph like edeno/d3-save-svg does.

Agree, it would be good to have an option to export both pure graph data as a JSON/XML and the image representation in SVG. I am not sure whether I will be able to work on it any time soon, but any PRs with such work are very welcome.
(actually, it is even easier now than it will be in the future since extension currently uses SVG as a rendering layer but there is ongoing work to migrate it to canvas/WebGL; nothing relevant, just mentioning it for the record)

Clickable embed for UX/UI Nav on live site.


Lab 8 & 9 - JSON puppy and LevelDB

Fixes #103, Fixes #104 & Fixes #112
Changes proposed in this PR:

Update project structure
Add JSON tags to Puppy
Add Storer implementation with LevelDB
Add command line flag -d FILE with long form --data FILE using kingpin.v2
A file containing JSON array of Puppies can be passed as an argument with -d or --data flag

Codecov Report

Merging #118 into master will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff          @@
##           master   #118   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files          35     35
  Lines         475    475
=====================================
  Hits          475    475

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b6f319...6749263. Read the comment docs.

@anzboi Thanks for the review. I have updated SyncStore and LevelDB implementations with locking to prevent race conditions and renamed tests to generic names.

@anzdaddy Thank you for reviewing the changes. Now that the PR is already merged how do i update the code with changes suggested? Shall i revert the changes with a new PR as github suggests or open a new PR with only the suggested changes?

I would open a new PR with just the suggested changes. Once things are merged to master, there's no real going back. You can revert commits, but you should save that for when things have gone quite wrong.
Iteratively refining your work over multiple PRs works fine as a model for software development. The main thing to keep in mind is that ideally everything in the repo continues to work after every commit  - i.e. you don't split work over two commits where the repo is broken after the first commit. Doing this correctly means you can use git bisect to identify bugs as git bisect works best when you can checkout any commit and run the tests and those tests pass.


v-else and v-for can't be used together

Version
2.5.13
Reproduction link
https://jsfiddle.net/50wL7mdz/95639/?utm_source=website&utm_medium=embed&utm_campaign=50wL7mdz
Steps to reproduce
v-else and v-for can't be used together
What is expected?
Rendering results

else-1
else-2
else-3
else-4
else-5
else-6
else-7
else-8
else-9
else-10

What is actually happening?
Rendering results
undefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefinedundefined

This is expected. v-for has higher priority than conditionals like v-if or v-show. If you put one on the same element, it will be executed for each item in the v-for loop.
Use a template tag:
<template v-else>

else-{{b}}

</template>

https://vuejs.org/v2/style-guide/#Avoid-v-if-with-v-for-essential


Opened article disappears after feed update

This was found by @kurnevsky.
Steps to reproduce:

Start full feed update
Open any feed
Open the last article in the feed

Expected result:
The article stays opened until I return to feed or items view.
What actually happens:
Once feed update finishes and if current feed received new items, the article disappears; counts on the top show absurd numbers (Article ''<PHONE_NUMBER> unread, 0 total)). When I quit to items view, the article I was reading isn't marked as read (even though it was unread when I opened it).
I successfully reproduced this on 2.9 and HEAD.

Moving this to 2.11 milestone to unblock 2.10 release. (Fixing this will require quite a bit of refactoring, something I don't want to get into right before a release. Sorry.)


Tracking issue - remove deprecated code

This is a tracking issue for the removal of deprecated code.

[ ] Remove support for embedded-hal v0.2
[ ] Remove deprecated items in sercom::dma
[ ] Optionnally, when #635 lands, remove the spi_future module - Would appreciate @bradleyharden's input on this.

I wouldn't remove embedded-hal v0.2 just yet. It can still be useful. Maybe hiding it behind a feature for now would be enogh.

Fine by me - out of curiosity, what would you consider a good timeline to start removing it?

I just looked into it further and I think it's not as bad as I thought. I knew some things were missing in v1, because they couldn't settle on an API. But I think they were sufficiently useless in their current state that they probably weren't used anyway.
Some drivers might still use v0.2, but those could be easily adapted (if they are still maintained...) or you could use embedded_hal_compat.
So it's probably safe to remove v0.2 with the next minor version bump.


SDK changes incorrectly/unexpectedly

Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug or Question
Enter Task Name: DotNetCoreCLI@2
Environment

Server - Azure Pipelines

malibu00 / Emmons / Emmons-[AI+R]-Security-CI .NET 6 / #3.2.22214.5811

Agent - Hosted

Pool: Azure Pipelines
Image: windows-2022
Agent: Azure Pipelines 2

Issue Description
Different instances of this task are running different versions of dotnet.exe, even though the settings are the same and only one SDK is expected for the entire pipeline.
All *proj files start out this same way:
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net6.0</TargetFramework>
  </PropertyGroup>

Compare the task dotnet build and dotnet publish TesApi and TriggerService
yaml:
steps:
- task: DotNetCoreCLI@2
  displayName: 'dotnet build'
  inputs:
    arguments: '-c release -o $(Build.ArtifactStagingDirectory)'

Beginning of log:
Starting: dotnet build
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.202.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
C:\Windows\system32\chcp.com 65001
Active code page: 65001
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
"C:\Program Files\dotnet\dotnet.exe" build "-dl:CentralLogger,\"D:\a\_tasks\DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b\2.202.0\dotnet-build-helpers\Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll\"*ForwardingLogger,\"D:\a\_tasks\DotNetCoreCLI_5541a522-603c-47ad-91fc-a4b1d163081b\2.202.0\dotnet-build-helpers\Microsoft.TeamFoundation.DistributedTask.MSBuild.Logger.dll\"" -c release -o D:\a\1\a
Microsoft (R) Build Engine version 17.2.0+41abc5629 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

to
yaml:
steps:
- task: DotNetCoreCLI@2
  displayName: 'dotnet publish TesApi and TriggerService'
  inputs:
    command: publish
    publishWebProjects: false
    projects: |
     **/TesApi.Web.csproj
     **/TriggerService.csproj

    arguments: '-r linux-x64 -c release -o $(Build.ArtifactStagingDirectory)'
    zipAfterPublish: false

Beginning of log:
Starting: dotnet publish TesApi and TriggerService
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.202.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
C:\Windows\system32\chcp.com 65001
Active code page: 65001
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.
C:\hostedtoolcache\windows\dotnet\dotnet.exe publish D:\a\1\s\src\TesApi.Web\TesApi.Web.csproj -r linux-x64 -c release -o D:\a\1\a\TesApi.Web

Welcome to .NET Core 3.1!
---------------------
SDK Version: 3.1.201

Telemetry
---------
The .NET Core tools collect usage data in order to help us improve your experience. The data is anonymous. It is collected by Microsoft and shared with the community. You can opt-out of telemetry by setting the DOTNET_CLI_TELEMETRY_OPTOUT environment variable to '1' or 'true' using your favorite shell.

Read more about .NET Core CLI Tools telemetry: https://aka.ms/dotnet-cli-telemetry

----------------
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Find out what's new: https://aka.ms/dotnet-whats-new
Learn about the installed HTTPS developer cert: https://aka.ms/aspnet-core-https
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli-docs
Write your first app: https://aka.ms/first-net-core-app
--------------------------------------------------------------------------------------
Microsoft (R) Build Engine version 16.5.0+d4cbfca49 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

Task logs
See above

Turns out adding the UseDotNet@2 task to the pipeline causes the desired SDK to be used by both tasks, but it doesn't alter the fact that both invocations of the exact same task-name/version employ two different paths to the dotnet binary. It's completely unexpected that two subsequent invocations of the exact same task would result in different binaries being run.

Adding the UseDotNet@2 task did not resolve this issue for me. The DotNetCoreCLI@2 seems to always use an older version of the CLI. The project in question here was built with Azure Function Core Tools. The publish task works when run as a script, but fails when run using DotNetCoreCLI@2
yaml:
steps:
- task: UseDotNet@2
  inputs:
    version: '6.x'

- task: DotNetCoreCLI@2
  inputs:
    command: build

- script: |
    dotnet publish
  continueOnError: true

- task: DotNetCoreCLI@2
  inputs:
    command: publish
  continueOnError: true

Start of logs for both the build and publish tasks using the DotNetCoreCLI@2 task:
==============================================================================
Task         : .NET Core
Description  : Build, test, package, or publish a dotnet application, or run a custom dotnet command
Version      : 2.208.1
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/build/dotnet-core-cli
==============================================================================
Info: .NET Core SDK/runtime 2.2 and 3.0 are now End of Life(EOL) and have been removed from all hosted agents. If you're using these SDK/runtimes on hosted agents, kindly upgrade to newer versions which are not EOL, or else use UseDotNet task to install the required version.

Commenting so this won't go state. I am encountering this as well.

In actions/runner-images#7218 it was discovered that the order of directories in PATH seems to change randomly with NO changes to any pipeline job anything. That issue was closed as a duplicate of another issue that doesn't appear to have anything to do with PATH, but that's a soapbox for another day.
I think it would be prudent to consider that PATH is suddenly changing between builds with no changes that should warrant any change to PATH. Furthermore, it appears that most of the pipeline components look for their binaries via PATH and never verify that the correct binary is actually invoked.

Commenting so this won't go state. This needs attention from someone in Azure.

Commenting again so this won't go stale.
On Fri, Mar 15, 2024, 9:06 p.m. github-actions[bot] <
@.***> wrote:

This issue is stale because it has been open for 180 days with no
activity. Remove the stale label or comment on the issue otherwise this
will be closed in 5 days
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/azure-pipelines-tasks/issues/16683#issuecomment-2001343868,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AASCFEUL2Z7W7RUH4K6K5VLYYOZMFAVCNFSM55K3TPUKU5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBQGEZTIMZYGY4A
.

Commenting again to keep the thread active.

I'm seeing the same. Commenting to keep this fresh.


indication and response seems missing 2 bytes

demo code:
var ModbusTcpServer = require("nodbus-plus").ModbusTcpServer;
var slave = new ModbusTcpServer(8502);

slave.on("error", function (err) {
  console.log(err);
});

slave.on("connection", function (socket) {
  console.log("connection");
});

slave.on("connection-closed", function (socket) {
  console.log("connection-closed " + socket.remoteAddress + " disconnected");
});

slave.on("client-disconnect", function (socket) {
  console.log("client-disconnect " + socket.remoteAddress + " disconnected");
});

slave.on("listening", function (port) {
  console.log("listening ", port);
});

slave.on("closed", function () {
  console.log("server closed");
});

slave.on("indication", function (adubuffer) {
  console.log("Indication Recieved");
  console.log(adubuffer);
});

slave.on("values", function (reference, address) {
  console.log(
    "data write on: " + (reference == "0x" ? "coils" : "holding registers")
  );
  address.forEach(function (value, key) {
    console.log(key + ": " + value);
  });
});

slave.on("response", function (resp) {
  console.log("response", resp);
});

slave.on("modbus_exeption", function (reason) {
  console.log("modbus_exeption, reason ", reason);
});

slave.Start();

sent hex from modbuspoll

i think maybe crc is missing, and could you please provide full hex and part of hex which is the same as modbuspoll? thank you very much!

Modbus TCP protocol don't use CRC or LRC on his frame. Instead use a Header, the first 7 bytes. The byte 6 is the byte count with the number of byte following. The indication event get the complete data frame received from the master.


Include full callstack with log entries

See how full callstacks can be included with log entries. Do they have to be part of the text message? can they be included as part of the json payload (so only visible when expanding log entries in the web UI)? If part of json payload, will searches in the web UI also search over these?

Implemented with bc814092ef479c53103dbedc1fc3ebebe3e86ca9.
Full callstacks can be included. If they are part of a multiline text message, with formatting like Exception.ToString()'s output, then Stackdriver Logging will automatically forward such errors to Stackdriver Error Reporting. It would also be possible to put the callstacks into the json payload, but the benefit of that appears quite small.


Slash-commands from non-enterprise workspaces trigger wrong "installationStore" method

Description
Slash-commands from non-enterprise workspaces trigger the fetchOrgInstallation method of a custom installationStore instead of using fetchInstallation.
This happens because slash-commands are of content type application/x-www-form-urlencoded (docs) and the is_enterprise_install property, which determines if the fetchInstallation or the fetchOrgInstallation, is a boolean. The resulting JSON has isEnterpriseInstall: 'false', which evaluates true because 'false' is a string, not a boolean.
Parsing the x HTTP body happens here and doesn't take boolean values properly into account:
https://github.com/slackapi/bolt-js/blob/fd35a5c3905b252df7ff53ae2f2cc890c80e13e7/src/ExpressReceiver.ts#L358
What type of issue is this? (place an x in one of the [ ])

[x] bug
[ ] enhancement (feature request)
[ ] question
[ ] documentation related
[ ] example code related
[ ] testing related
[ ] discussion

Requirements (place an x in each of the [ ])

[x] I've read and understood the Contributing guidelines and have done my best effort to follow them.
[x] I've read and agree to the Code of Conduct.
[x] I've searched for any related issues and avoided creating a duplicate issue.

Bug Report
Filling out the following details about bugs will help us solve your issue sooner.
Reproducible in:
package version: 2.6.1
node version: v14.15.4
OS version(s): macOS 10.14.6
Steps to reproduce:

add a slash command handler to a bolt js app
add OAuth handling with a custom installationStore providing all 4 functions
trigger that slash-command from a non-enterprise workspace

Expected result:
The fetchInstallation method of the installationStore is called.
Actual result:
The fetchOrgInstallation method of the installationStore is called.
Note:
I know that the whole separation between fetchOrgInstallation and fetchInstallation will be removed in the next major release, but I don't know if there are other boolean properties sent in an urlencoded body that need to be parsed separately. So maybe that's something you need to look into.
Thank you for this awesome framework and have a great day!

Hey @simonme,
This was reported and fixed recently in https://github.com/slackapi/bolt-js/issues/737. We are planning on doing a release tomorrow with the fix.

Hey @simonme,
This was reported and fixed recently in https://github.com/slackapi/bolt-js/issues/737. We are planning on doing a release tomorrow with the fix.


Publish updates from master

Proposed changes

Unreleased project version (optional)

Related issues (optional)

@jerae-duffin @StefanScherer PTAL


Enable unit tests for circle CI

@ayj FYI. Thanks for the pointer to our testing docs for minikube options.
NONE

I didn't look too closely at the original CircleCI PR. Was there an issue/doc with motivation, long term plans, etc. for this approach?

Its just something for non google folks. Having a commonly understood test infrastructure, and something that others can debug/view easily. Its fast, and low overhead as well, as you can see from the PR (just one circle ci config), and easily reproducible locally. Basically, a very boring test infrastructure :).

And its a backup. The nice thing about this is its completely isolated. One VM per test. We don't have to worry about resources leaking, or underlying infrastructure running out of IPs, etc.


feat: Added default example values to schemas

This little PR adds default example values to schemas by adding example parameters in classes from schemas.py.
I don't know whether or not to add examples for the fields ids, dates and token. What do you think?
Also, I am wondering if we should add description to  field.

Feedbacks are welcome

I think we are good now.


Add tree builder

We should add a real treebuilder and lol-html like api on top of that to this crate. As part of that we need to move the tokenizer into a submodule and rework the readme

In the meantime #11 should be fixed


(DONT MERGE) Actions: Skeleton to run tests from fork

NOTE: DO NOT MERGE
Issue link / number:
N/A
What changes did you make?
Enabled secrets access to workflow tests from public pull requests.
Uses pull_request_target but only if the workflow triggering actor is from Chayn.
This means pull_request_target will not grant secret access to anyone who makes a pull request.
Instead, all public forks triggered by Chayn team members will have access to secrets, even on public forks.
This PR contains the basic functionality to isolate basic testing before opening another PR with full implementation.
Why did you make the changes?
To save developer time by removing the need to run tests locally for contributor pull requests. With this, devs can conveniently run tests for public pull requests in the UI.

Closing this PR because contributors still be able to run workflows even if they fail before secrets access. However, this is a problem, as all re-runs of workflows will use the initial run’s actor for privilege evaluation. (GitHub Changelog 2022)
Instead we will isolate build & test workflows requiring secret access out of public contributor PRs.
Chayn team, view internal tech community docs for more details.


circuit.library.graph_state needs testing

The object GraphState (in qiskit/circuit/library/graph_state.py) needs testing.  No line is currently coveraged.

Hi, I'd be interested in contributing if this issue is still available. This would be my first issue so I'm not completely sure where to start!

Hi, I would like to work on this issue, would it be possible to assign this to me if it is not being worked on?

Hi!
Great that you want to contribute 🎉
So the task would be adding tests for the GraphState circuit. This concept is introduced in this paper which you can look up for some context.
A test should make sure that the implementation in Qiskit works (i.e. it runs w/o errors) and is correct (i.e. the output is correct).
Ideally for such a circuit we would check the implementation on a theoretical level. This means we're checking the Qiskit implementation against what the maths says. This can e.g. be done by computing the unitary the circuit implements and then simulating the circuit in Qiskit.
This is done e.g. for the HiddenLinearFunction:
https://github.com/Qiskit/qiskit-terra/blob/8771e595a67e584f7ce1bac2d70db84516cd3432/test/python/circuit/test_library.py#L150
and the QFT:
https://github.com/Qiskit/qiskit-terra/blob/8771e595a67e584f7ce1bac2d70db84516cd3432/test/python/circuit/test_library.py#L346
To do this you could have a look at Eq. (9) and following in the paper, where it describes what state the circuit builds (called |G> in the paper). For this check we would compute the matrix that prepares |G> as described in the paper and compare it to the GraphState.
Another simpler way, which would already be a good start, is to compare the circuit to  a hardcoded reference circuit. This is e.g. done for the IQP circuit:
https://github.com/Qiskit/qiskit-terra/blob/8771e595a67e584f7ce1bac2d70db84516cd3432/test/python/circuit/test_library.py#L190
You can check this reference files and let me know if you have any questions! What's the tactic you would like to pursue?

Hi @Cryoris!
Thank you so much for going into such details and explaining this so clearly! Your description has provided more than enough background for me to work on this 🙂
I would like to make a check on a theoretical level, it seems quite straightforward to calculate the matrix by hand for a particular graph. Would it be fine if I pick a smaller graph that can be implemented by up to 4 qubits, e.g., the graph No. 3 in FIG. 4 of the paper? I am also considering creating multiple tests for graphs of various sizes, but let me know if that would be too much.
Thank you again for guidance!

Hi again!
I just wanted to give a quick update about this issue. I have looked into testing the graph states a bit more and I have come up with a piece of code that creates the expected unitary for a given adjacency matrix. I thought this might be a relevant update, because my previous point from the above comment that I would be calculating the matrix by hand is not valid anymore. I think it makes more sense to numerically calculate the expected matrix based on the adjacency matrix, since that would allow to change the test by simply changing the adjacency matrix. I can see that this is the approach used in other test cases, e.g., for HiddenLinearFunction. This was most likely the approach that was expected to begin with, so I am sorry to have figured that out just now 🙂
As a quick outline, the way I calculate the expected matrix from the adjacency matrix is quite simple. First, I create the matrix for Hadamard gates for the number of qubits extracted from the adjacency matrix. Secondly, I create the matrices for the CZ gates based on the adjacency matrix. This step is slightly more involved, since I need to get the CZ gate for any given number of qubits and control-target connections. I do that by looping through an identity matrix and setting the value of the diagonal elements to -1 where both the control and target are in state |1>. Lastly, all the matrices are multiplied together to get the expected unitary. I have tested this approach and the results look good.
I would now like to work on adding the mentioned code to test/python/circuit/test_library.py. Once the code is properly implemented and I am satisfied with the addition, I would like to create a pull request, if that would be fine 🙂

Hi @mantcep,
sounds great that you found a way to compute the generic unitary transformation that creates the graph state. A fixed comparison would also be good, but if we can do it generically that's of course better.
It is easier to comment on the code if we can see it, so feel free to open a pull request already now and name it [WIP] Your title to show that it is still work in progress. Instead of looping through all indices you might also be able to compute the indices of control and target qubit and directly check their states. But I'll give more feedback once you opened the PR!
Sounds like a good plan 👍

Hi @Cryoris,
I have created a pull request as suggested. I tried looking into a better way of getting the CZ gate, but I could not think of anything straightforward. Let me know what you think, I could put more time into vectorizing the loop over the identity matrix.
Thank you for your time! 🙂