Add pre-commit for automatic formatting

This PR adds a pre-commit config to run clang formatting. I checked the CONTRIBUTING.md and didn't see any format requirement. For anyone not familiar with pre-commit, here is a helpful tutorial:  Check formatting locally and on GitHub Actions with pre-commit
In short, pre-commit can be installed via: pip install pre-commit
and ran using: pre-commit run --all
The formatting is setup to run as part of the CI.

Hi @pac48
I need a little more context in this PR. What are the benefits of using clang formatting instead of the ROS 2 standard?
Best
Francisco

Hi @pac48
I need a little more context in this PR. What are the benefits of using clang formatting instead of the ROS 2 standard?
Best Francisco

Sorry for not elaborating, let me explain my issue. I opened a PR Add Support for Contingency Planning, which has a huge diff because I used a different format. That is no problem, the code can be reformatted, so I thought adding clang automatic formatting as part of CI would be nice.
One advantage of the clang-format over the ament based tools is the de facto standard in C++. It is a consistent and powerful tool for formatting. The real downside to clang-format is that it is kind of annoying to run locally. But that is where pre-commit comes in. It is a python script that automates running tools like clang-format locally for developers and can also be used in CI to verify the formatting stays consistent. With pre-commit, you can also verify the formatting of python files, spell check, check YAML formatting, etc. The list goes on. When applying to the this repo, there were actually quite a few changes recommended by the python formater "black". There were also many spelling mistakes that were caught suggestions recommended.
The biggest problem with clang-format is that it can't be applied to existing ROS 2 code bases because there are some small formatting exceptions it does not allow. See here for a discussion.
I don't have a strong opinion on which formater should be sued, although it should be within the ROS code style guidelines of course. With that said, here is a branch from master after running ament_uncrustify --reformat
run-ament-reformat. You can see there is a huge diff still. I think having some automatic formatting will save headaches in the future.
Is there some other tool (besides clang-format and ament_uncrustify) that you recommend?

I have not forgotten this PR. I need time to think deeply about this.


error at modules.ini polybar

Sorry for open another issue but, In ~/.config/bspwm/polybar/modules/modules.ini at the line 565-571:
[module/powermenu]
type = custom/text
content = ""
content-font = 2
content-background = ${colors.surface0}
content-foreground = ${colors.red}
click-left = $HOME/.config/bspwm/apps/rofi/bin/powermenu <--- this ins't in that directory, there is at  $HOME/.config/bspwm/apps/rofi/powermenu

Fixed this, thanks for notifying me. I don't use some of these modules so I didn't update them but this will be fixed in the next commit.


Discover page - search fails on the term 2015 on www

Oddly the search for 2015 appears to fail...

Valid: https://www.hydroshare.org/search/?q=2013
Valid: https://www.hydroshare.org/search/?q=2014
Error: https://www.hydroshare.org/search/?q=2015
Valid: https://www.hydroshare.org/search/?q=2016

Update - 12/29 - Based on Ray's note of being the current year, the search term dec also fails

Valid: https://www.hydroshare.org/search/?q=oct
Valid: https://www.hydroshare.org/search/?q=nov
Error: https://www.hydroshare.org/search/?q=dec
Valid: https://www.hydroshare.org/search/?q=jan

Will be curious to check this again on 1/1/2016 to see if the error shifts to the new month and year.

Noting that 2015 s is the current year.  Further debug on Friday Jan 1 2016 to see if 2015 works and 2016 fails?  Also, should add check to see if entered year is greater than the current year.

Also, should add check to see if entered year is greater than the current year.

We probably do not want to exclude any search terms as they could be relevant to other data entered by the user. A future year for example could refer to an abstract note for future appearance/publication or even a value of something other than a date. (i.e. After taking 2020 samples of XXX it was discovered that...)

Noting that 2015 s is the current year. Further debug on Friday Jan 1 2016 to see if 2015 works and 2016 fails? Also, should add check to see if entered year is greater than the current year.

Still see the same error pattern as before, so it doesn't seem to be associated with the current month or current year.

@rayi113, @hyi, Out of curiosity I issued a rebuild_index call directly on www and that seemed to resolve our issue. I'm wondering if our current update_index is not sufficient in all cases, but rather a periodic rebuild_index is needed... Thoughts?

@mjstealey @rayi113 I am not sure why rebuild_index is needed to fix this issue. Perhaps some scheduled update_index call failed to some degree, which leads solr index in a somewhat not completely correct state, and rebuild_index kinds of reset solr index to an initial correct state. Not sure, perhaps @alvacouch @xuezhaokun @ironfroggy can have better explanations.

@hyi, @rayi113, I'm not up on solr / haystack enough to know when it's best to use rebuild_index vs update_index... rebuild_index strikes me as kind of a nuclear option when all else fails based on the warning it gives.
WARNING: This will irreparably remove EVERYTHING from your search index in connection 'default'.
Your choices after this are to restore from backups or rebuild via the `rebuild_index` command.
Are you sure you wish to continue? [y/N]

@hyi - Adding snippet from hipchat with @ironfroggy for reference

@alvacouch, @y-lu - This is the issue referred to on the 1/6/16 team call. As noted a rebuild_index did the trick. Had a brief discussion with @ironfroggy which is summarized above ^^^

@xuezhaokun @alvacouch Can you evaluate and see whether this issue has been resolved by your real-time solr index update upon resource deletion? If it has been resolved, you can close this issue unless there is more work to be done.

fixed on www


Refactor single-sample, germline CWL pipeline as constitutional variant pipeline.

Remove gVCF outputs, they are not needed. Ensure a final Whole Genome variant call file exists. contact Dave Spencer for details.

@dufeiyu are these changes ready for a pull request back to our repo and/or master branch? I recall you mentioned making and testing some changes for the CLE.

copy/paste from a slack thread:

I'm going to throw out a question. brainstorming here.... if the end goal is a single-sample VCF, why are we running GenotypeGVCFs at all?
We've decided to make two separate workflows 1) supports joint-genotyping (GVCF output or GenomicsDB) 2) a single-sample "germline" or "constitutional" workflow
for #2, can't we simply run HaplotypeCaller in VCF mode instead of GVCF?
Here is a link to a Broad WDL where the make_gvcf input controls the "mode" of the workflow: https://github.com/gatk-workflows/gatk4-genome-processing-pipeline/blob/master/tasks/VariantCalling.wdl#L25


Consider cloud IDE with cloud dev container

Currently the recommended approach to development is to use an IDE installed on a local workstation.   In the near future development may be mostly happening in a web browser.   I already make minor changes such as documentation changes routinely in-browser on github.com.
Local workstation development requires users to install an IDE and other tooling, plus patch and update constantly.    Ideally this is repeated similarly for each member of a team and in each office of each person (I currently must patch IntelliJ, Docker, Java, and Gradle on my home office PC, work office PC, and travel Macbook).   A docker compose quick start is great, but running it in a dev container is even better as it reduces the difficulty of entry for interested parties even lower, especially great for potential contributors.
Up and coming cloud development solutions include GitHub codespaces and IntelliJ spaces plus dozens of others.   Cost is a consideration though as free tiers have limited hosted resources.

Tried out codespaces.    It's pretty cool as everything happens in-browser via visual studio code web edition.   The built-in terminal is actually pretty good (you can create multiple tabs, run docker compose up in one while running docker exec in another, and htop in another and gradlew build in another.).   This project needs the 8GB container (as opposed to default 4GB) else runs out of memory and fails to start the oracle database container when executing docker compose up (inside the dev container).   Would probably be most useful if a dev container prebuild was created.  However, you have to be an organization admin to do that (I can only create prebuilds for my personal projects).   Also, the docker-in-docker has some quirks that would need to be worked around - in this project our app redirects users on login to keycloak, and that doesn't work in this dev container env because the host names are created on the fly and randomly assigned whereas the current local workstation development relies on fixed localhost name and fixed ports for the redirect to https and redirect to keycloak and redirect back to demo app.

Redirect issue persists, but you can use docker in sibling mode and manually pre-build image: https://github.com/orgs/community/discussions/113197


fut: add better text

Change Log
Added Lombok Dependency

Introduces Lombok to simplify error handling and reduce boilerplate. This pull request benefits from the removal of redundant code, enhancing maintainability.

Enhanced Formatting Capabilities

Integrated improved formatting options, including color, links, and a smart markdown feature supporting command execution on click. For more details, see MineDown on GitHub.

Refined Code with DRY Principles

Implemented a formatter that centralizes logic, data formatting, and empty value handling. This change streamlines the codebase for efficiency and readability.

Shifted from Broadcasting to Private Messages

Broadcasting has been disabled in favor of private messages due to:

Better alignment with my needs.
Discovery that BaseComponent may not support broadcasting (further investigation required).
Potential for incorporating broadcasting differently, e.g., through configuration settings:

on-login:
   private: 'Private message'
   public: 'Public message'

Utilization of event.setCancelled(true) to prevent the display of both custom and vanilla join messages.

Removed Item Giving Feature

Concluded that item giving is more suited to a kit plugin rather than the Welcomer plugin, aiming to streamline the plugin's purpose.

@micfun123, please review these changes and provide feedback. I look forward to the potential integration of these updates.


bug: When meltano.yml is empty, no error message is printed. Rather, it just mentions to reach out to community

Meltano Version
2.19.0
Python Version
3.9
Bug scope
CLI (options, error messages, logging, etc.)
Operating System
Windows - WSL(Ubuntu)
Description
when meltano.yml is empty, cli(meltano.cli.__init__.py: 105) raises EmptyMeltanoFileException exception whenever we try to run any command such as meltano add or meltano ui. But, since there's no exception message, it just prints the troubleshooting message and blank lines as follows
Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Code
No response

@rawwar thanks for the bug report. Seems like an easy fix to throw some text on the command line there. I'll add the Accepting PRs label, but perhaps @meltano/engineering will pick it up too!
Out of curiosity, how are you winding up with an empty meltano.yml?

🤦 I didn't see the PR you opened as well! Thanks!

@rawwar thanks for the bug report. Seems like an easy fix to throw some text on the command line there. I'll add the Accepting PRs label, but perhaps @meltano/engineering will pick it up too!
Out of curiosity, how are you winding up with an empty meltano.yml?

I was actually working on some other thing. It kept complaining no meltano.yml.. so, I just created an empty file and forgot about it. I kept seeing empty error messages after a while.. when I tried to debug.. found the issue


[draft](tablet scheduler) fix higher priority tablet add failed due to pending queue full

When tablet checker search for unhealth tablets,  it will search table one by one.  If the pending queue is full,  then the later search tablets couldn't add into the pending queue,  even if they are higher priorty.  So we use a minmax queue to holding the tablets. If a tablet is higher priority and the pending queue is full,  it will envit the lowest priority tablet in the queue, then add this tablet into queue;

run buildall

run buildall

run buildall

run buildall

run p0


Saving incorrectly updates state if switching to XML tab at least once

Version: 1.0.0

Cannot reproduce it. Might have been fixed with the changes in importing and exporting when switching views.


XBOX One controller button mapping is backwards.

XBOX One controller button mapping is backwards.
A is B and B is A, X is Y and Y is X.
Apart from this it's not very intuitive on when/how to get a controller to connect to the device. Is it possible to enable the LED to "flash" or indicate when it's connecting a device? I am thinking of installing a blue LED on the GPIO and having it enabled for connections.

There is no remap in the Switch menu possible

Sorry for the delayed response.

XBOX One controller button mapping is backwards.
This can be easily changed inside pico_switch_platform_init

Is it possible to enable the LED to "flash" or indicate when it's connecting a device?
Yes, the led can be toggled on/of with this line

cyw43_arch_gpio_put(CYW43_WL_GPIO_LED_PIN, 1);

The legacy binary of the program has led flashing that indicates status, I removed it while debugging the pico crashing and forgot to re-add it.

I hope to compile it some day. But I'm waiting for the DS5 fix first to avond to use the legacy version. Or to make a separate DS5 version.
@juan518munoz

Legacy and current version are funtionally the same, the only difference is in it's codebase.

Legacy uses bluepad32 as a directory and is heavely modified.
Current version uses bluepad32 as a submodule so updating it is easier.

@DaPiMan I compiled a version with the buttons in correct order
PicoSwitchWGA-xbox.zip
@juan518munoz there are no instructions on how to compile legacy version sadly

Legacy version is based of this commit, you can git reset into it so you can compile it.

Apparently my compiled firmware does not work. I removed the link and will try to recompile it.

I recompiled the button switch. Tested with my Xbox Elite 2 controller and I played over 50 mins fine.
PicoSwitchWGA-xboxremap.zip
Dualshock 4 indeed has a problem, you can change the color on the controller and there are timeouts or connection problems.
I will try with the reset option and recompile. Maybe the Dualshock 4 problem is the same as the Dualshock 5 (Dualsense)?

The legacy version must be compiled with ./build.sh


refactor: make KernelTestBehaviour trait methods static

I created no Changelog yet, it should first be discussed if this change could have any negative side effects.
It's still possible to call $this->getContainer() in classes which have use KernelTestBehaviour;.
So the only sideeffect I can think of is if someone wrote an own trait which specify either
abstract protected function getKernel(): KernelInterface; or
abstract protected function getContainer(): ContainerInterface;
the types don't match anymore, but still this should have no negative impact.
Edit: ~I converted this to a draft. Until https://github.com/shopware/platform/pull/2608#issuecomment-1188939079 is resolved this PR is useless.~
Edit 2: I found a workaround. Maybe this can be implemented in the Shopware core? See https://github.com/shopware/platform/pull/2608#issuecomment-1189112331
1. Why is this change necessary?
This change makes the Shopware KernelTestBehaviour compatible with Symfony\Bundle\FrameworkBundle\Test\MailerAssertionsTrait
The MailerAssertionsTrait was introduced with Symfony 4.4 https://symfony.com/blog/new-in-symfony-4-4-phpunit-assertions-for-email-messages
2. What does this change do, exactly?
This changes all getKernel and getContainer method implementations and abstract definitions to static methods.
3. Describe each step to reproduce the issue or behaviour.
Add use MailerAssertionsTrait; to your testclass and try to call any of the methods of it. For example $this->getMailerMessages();
This results in
Error : Non-static method Vcore\MyPlugin\Test\Error\ErrorHandlerTest::getContainer() cannot be called statically
4. Please link to the relevant issues (if any).
5. Checklist

[ ] I have written tests and verified that they fail without my change
[ ] I have created a changelog file with all necessary information about my changes
[ ] I have written or adjusted the documentation according to my changes
[ ] This change has comments for package types, values, functions, and non-obvious lines of code
[ ] I have read the contribution requirements and fulfil them.

Now the functions are callable but the assertions still don't work.
The problem is that the assertions depend on the Symfony\Component\Mailer\EventListener\MessageLoggerListener which listens for the Symfony\Component\Mailer\Event\MessageEvent.
Unfortunately there is no message bus defined while testing, so the MessageEvent is never dispatched because the send function returns earlier: https://github.com/symfony/mailer/blob/57384499f838a00776fb1d9e58f7e6e1a706cf71/Mailer.php#L42
Is there any workaround for that?

I can now successfully use $this->getMailerMessages(); when I add the mailer.mailer service in my services_test.xml like this:
<service id="mailer.mailer" class="Symfony\Component\Mailer\Mailer">
    <argument type="service" id="mailer.transports" />
    <argument type="service" id="messenger.default_bus" />
    <argument type="service" id="debug.event_dispatcher" on-invalid="ignore" />
</service>

The important line is <argument type="service" id="messenger.default_bus" />. By default <argument>null</argument> is passed while testing instead.
Is there a good solution to implement this in the Shopware core so testing and mail assertions just work like they do in Symfony?

Hey,
I will close this PR as it's inactive for a while. Feel free to reopen a new one if you have finish with your changes.

Hey @RafaelKr, it's fine for us when you change them all to static. Please make a new PR as we changed A LOT in the trunk right now also updated to Symfony 6.2 and so.

Hey @shyim I created #2928. It's ready to merge, it's only missing a Jira issue id.


Clojure language runner

As I'm using clojure for large and more and more projects, it would be fine, to have a clojure language runner also.
Is someone else interested in?
Some chance to implement such a runner in a team?

@craftybones You were talking about writing a Clojure runner for Gauge. You may want to get in touch with @jerger.

@jerger I'm certainly interested.

:-) To get in touch u can use my email - it's visible in my github profile ...

closing as stale, reopen the issue if needed

I'd be very happy to use Gauge with Clojure too !


add 17.07 changelog

ping @andrewhsu anything to add packaging wise ?
ping @thaJeztah

@vieux we need to add the telemetry plugin, it will be installed, but disabled by default.

@vieux for some reason it didn't push to your branch, but you can take this commit; https://github.com/thaJeztah/docker-ce/commit/19f70adcf6d907a8919f1d189846581e31b67b33

LGTM


LiveQuery build error = include of non-modular header inside framework module 'Bolts BFCancellationToken'

include of non-modular header inside framework module 'Bolts BFCancellationToken'
Hey everyone...
I just downloaded the Parse sample project, installed LiveQueries pod and nothing else. When I try to build it, it pops the two errors below.. What's wrong?
I'm using Xcode 7.3.1

Please report in the iOS-OSX-SDK repository, as it's not related to parse-server


Duplicate entries in public instance

It seems like there are some duplicate entries:
For example:
https://civic.genome.wustl.edu/#/events/genes/57/summary/variants/154/summary/evidence/409/summary
https://civic.genome.wustl.edu/#/events/genes/57/summary/variants/154/summary/evidence/410/summary
These two entries seem identical except that one is created by 'admin' and the other by 'rbilski'.

Wondering if this was somehow introduced when backfilling ownership of evidence items.

Yeah, these were introduced during the backfill. The duplicates have been removed - with precedence given to Evidence Items that had a "real" curator.


[new : feature]  support regsiter instance to etcd

Volunteer help
Hi Community
We already have the spi interface and already support zookeeper.
So there will be the following tasks：

[ ] new module register-instance-etcd
[ ] impl org.apache.shenyu.register.instance.api.ShenyuInstanceRegisterRepository
[ ] add unit test
[ ] add doc to shenyu-webiste

Hi, can I try this?


Add confirmation screen for registration to educate

If newbie
Educate that seat is not guaranteed

You are about to register as a newbie without previous reputation. We can not guarantee that you will get assigned for the next ceremony (link to FAQ:why have I not been assigned?). Please check your assignment status on the day before the ceremony to learn if and where your ceremony meetup will take place.

If endorsee
Educate that endorsement guarantees a seat but  is valid for this ceremony only

You have been endorsed by one of the bootstrappers of  as a trustworthy community member. This means you are guaranteed to be assigned if you register for this ceremony. The endorsement is only valid for this ceremony

If reputable
Educate that a seat is guaranteed but a no show will cause a loss of reputation and may downgrade to newbie

You have previously participated in a ceremony and therefore you are guaranteed to be assigned for this ceremony if you register. Caution: Should you register but not show up at the ceremony, your reputation will be reduced to newbie status.

If bootstrapper
You have bootstrapper status for community . Therefore, you are guaranteed to be assigned for the ceremony if you register.
if unused endorsement tickets > 0

You have  endorsement tickets left. Please consider endorsing new people to accelerate the growth of

also check #639

This should can be done in the onFinish callback of submitting the extrinsic:
https://github.com/encointer/encointer-wallet-flutter/blob/4cda4c5519cb15120ecd6efd39f89da1d935a59b/lib/service/tx/lib/src/submit_tx_wrappers.dart#L124
change that to something like:
  return submitTx(
    context,
    store,
    api,
    registerParticipantParams(store.encointer.chosenCid!, proof: await api.encointer.getProofOfAttendance()),
    onFinish: (BuildContext txPageContext, Map res) async {
      // Registering the participant burns the reputation.
      // Hence, we should fetch the new state afterwards.
      store.dataUpdate.setInvalidated();

      // new code here

      // get the updated participant status after registering
      final accountData = await api.encointer.getAggregatedAccountData();

      // show confimation screen (dialog might be better)
      showConfirmationDialog(context, accountData.personal!.participantType);
    },
  );

Simplified version for bootstrapper:

If you have endorsement tickets left, please use them to facilitate the growth of your community.


Exception response codes

What Exception response codes have you defined?
They do not appear conforming to the Modbus spec.
The library sourcecode uses magic values, but that keeps them unexplained.

I based the exception response codes on MODBUS Application Protocol Specification V1.1b3 for the function codes that this library supports, which are noted in the README.  I believe these are only function codes 1, 2, 3, and 4.
If you have found a discrepancy between the code and the specification, please indicate the lines of code in which the discrepancy occurs and the relevant document and page of the specification.


Cannot read TOC after rendering with contrib.toc_renderer

I just came across for surveying some CommonMark implementations in Python and noticed the TOC feature. It seems nice, but unfortunately I couldn't get it to work. Here is the minimal reproduction of this bug:
Setup:
git clone https://github.com/miyuchina/mistletoe
cd mistletoe
pip install -e

In this directory, run the following code:
from contrib.toc_renderer import TOCRenderer
from mistletoe import Document
with TOCRenderer() as r:
    r.render(Document('# aaa\n## bbb'))
print(r.toc)

The last line try to access the TOC tokens, raising an error:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/mistletoe/contrib/toc_renderer.py", line 45, in toc
    return List([build_list_item(heading) for heading in self._headings])
  File "/tmp/mistletoe/mistletoe/block_token.py", line 453, in __init__
    self.children = [ListItem(*match) for match in matches]
  File "/tmp/mistletoe/mistletoe/block_token.py", line 453, in <listcomp>
    self.children = [ListItem(*match) for match in matches]
TypeError: __init__() takes 4 positional arguments but 7 were given

7 is because # aaa is transformed to (- aaa\n), which has a length of 6, prepended with self.
Maybe the implementation changes so that block_token.ListItem now receives fixed number of arguments instead of plain str? Also note that the corresponding test:
https://github.com/miyuchina/mistletoe/blob/2cfe7446b975685f98837f9e40aaabcc0e270a79/test/test_contrib/test_toc_renderer.py#L43-L59
Does not test this feature thoroughly. It uses a mocked object and will succeed anyways.

Here is an ad-hoc patch to contrib.TOCRenderer:
      @property
      def toc(self):
#         (omitted)

-         return List([build_list_item(heading) for heading in self._headings])
+         lines = [build_list_item(heading) for heading in self._headings]
+         items = block_token.tokenize(lines)
+         return items[0]

Thanks for the report and the proposed fix.
I can confirm that surprisingly the unit tests really do not test the whole scenario. That's why they didn't reveal the error of passing quite different arguments into the List constructor than actually expected (list of strings vs. list of tuples).


HAMT transients: move mutable inner nodes when posible during removal

Follow up from #217
FYI @omer-s @harryhk

Codecov Report

Merging #216 (1897252) into master (95889a7) will decrease coverage by 0.38%.
The diff coverage is 70.37%.

@@            Coverage Diff             @@
##           master     #216      +/-   ##
==========================================
- Coverage   91.26%   90.87%   -0.39%
==========================================
  Files         114      114
  Lines       11026    11203     +177
==========================================
+ Hits        10063    10181     +118
- Misses        963     1022      +59

Impacted Files
Coverage Δ

immer/detail/rbts/node.hpp
83.12% <50.00%> (ø)

immer/detail/hamts/node.hpp
78.74% <55.45%> (-4.71%)
:arrow_down:

immer/detail/hamts/champ.hpp
85.28% <84.33%> (-0.38%)
:arrow_down:

test/map/generic.ipp
98.73% <93.33%> (-0.27%)
:arrow_down:

immer/detail/rbts/operations.hpp
90.29% <100.00%> (ø)

immer/detail/rbts/rbtree.hpp
96.47% <100.00%> (ø)

immer/map.hpp
97.80% <100.00%> (+0.04%)
:arrow_up:

immer/table.hpp
100.00% <100.00%> (ø)

immer/detail/util.hpp
81.81% <0.00%> (+0.56%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95889a7...1897252. Read the comment docs.


WATCH VIRAL TIKTOK VIDEO Sophie Rain Spiderman Video Tiktok Full Video

Sophie Rain Spiderman Video Tiktok Full Video
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
If you’ve been online lately, you’ve probably seen the buzz surrounding Sophie Raine’s SpiderMan video on Twitter. This unique creation by a promising director has captured the hearts of audiences all over the world. Today we take a look at the fascinating journey of Sophie Raine and her creative take on the iconic Spider-Man character. From initial concept to final execution, Sophie’s vision unfolds through dance and movement, giving us a new perspective on our favorite superhero.
The creative path of Sophie Raine: from director to creator of SpiderMan videos
Sophie Raine’s creative journey began as an aspiring filmmaker with a passion for storytelling. Raised in a family that appreciated the arts, Rain developed a love for film, comics, and drama from an early age. With the support of her parents, she developed her creativity and strived to make her mark in the digital space. While brainstorming a new project, Rain came up with the idea to create a unique and beloved Spider-Man character. Taking inspiration from fan theories and her creative vision, she came up with a video that would offer a new perspective on the superhero story through dance and movement.
➤ WATCH FULL VIDEO HERE
Sophie Rain Spiderman Video Tiktok Full Video
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
Turning vision into reality
Creating the Spider-Man video was a serious challenge for Sophie Raine. She faced numerous obstacles in taking her creative vision from concept to execution. One of the initial challenges was to assemble a talented team of dancers, choreographers, and a skilled production team that could bring her vision to life. Raine also sought sponsorship to support the joint venture, ensuring that the necessary resources were available for the ambitious project. Through careful planning and determination, she overcame these obstacles and successfully translated her artistic vision into a compelling Spider-Man video that resonated with audiences around the world.
Sophie Rain Spiderman video TikTok
➤ WATCH FULL VIDEO HERE
Sophie Rain Spiderman Video Tiktok Full Video
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
🔴 WATCH ORIGINAL VIDEO CLICK ➡ HERE
Sophie Raine’s Spider-Man video was born from an intense brainstorming session in which she wanted to depict the journey of her favorite superhero through dance and movement. Drawing inspiration from various fan theories and her creative vision, Rain sought to offer a fresh take on the iconic character. She wanted to create a video that would captivate viewers with its unique approach to storytelling and stunning visuals.
Rain’s passion for film, comics, and drama from an early age played a crucial role in shaping her artistic direction. Thanks to the support of her art-loving family, Raine had a supportive environment that allowed her creativity to flourish. This foundational influence paved the way for her innovative Spider-Man video concept.


'These' is used with plural words like Patterns

Descendents->Descendants

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @gshacklebolt
Your changes look fine, but you will need to create a new Pull Request and target the correct guide article. Your current changes target the mock-guide folder which is not used for production purposes.
I have closed this PR.
Thank you.


Bluetooth RCP firmware

Untested beyond creating builds.
This is the Bluetooth RCP firmware. Should be flashable once universal-silabs-flasher is patched to support the firmware type.

Might be a good idea to wait for Bluetooth version <IP_ADDRESS> as scheduled for release in Q3 of 2024 as part of Gecko SDK <IP_ADDRESS>?
https://www.silabs.com/developers/software-development-kit-release-notes
See indirectly related discussion here about Silabs Zigbee EmberZNet <IP_ADDRESS> and Silicon Labs Gecko SDK <IP_ADDRESS> announcements:

https://github.com/zigpy/zigpy/discussions/1414


[RNN] Keras LSTM converted to "While" OPs with hidden states manipulation - TFLite

1. System information

OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 11
TensorFlow installation (pip package or built from source): pip
TensorFlow library (version, if pip package or github SHA, if built from source): 2.15.0

2. Code
import numpy as np
import tensorflow as tf
from tensorflow.keras import Model
from tensorflow.keras.layers import Dense, LSTM

model_name = "1x_LSTM_64_float32"
input_length = 256

class SimpleModel(Model):
  def __init__(self, input_shape, hidden_size):
    super().__init__()

    self.lstm = LSTM(hidden_size, return_sequences = True, return_state=True, input_shape = [-1, input_shape] )

    self.d1 = Dense(1, input_shape = [-1, hidden_size])

  def call(self, x):

    x, h0, c0 = self.lstm(x)
    x = self.d1(x)

    return x#, tf.stack([h0, c0])

model = SimpleModel(input_length, 64)

out, states = model(tf.random.uniform([16,43,256]))

print(np.mean(out))

model_path = f"{model_name}.tf"

run_model = tf.function(lambda x: model(x))
BATCH_SIZE = 16
STEPS = 43
INPUT_SIZE = 256
concrete_func = run_model.get_concrete_function(
    tf.TensorSpec([BATCH_SIZE, STEPS, INPUT_SIZE], tf.float32))

model.save(model_path, save_format = 'tf', signatures=concrete_func)

converter = tf.lite.TFLiteConverter.from_saved_model(model_path)
converter.target_spec.supported_ops = [
  tf.lite.OpsSet.TFLITE_BUILTINS,
]

tflite_model = converter.convert()
open(f"{model_name}.tflite", "wb").write(tflite_model)

The previous model produces the following graph:

With a simple hidden states retention by modifying the following code:
class SimpleModel(Model):
  def __init__(self, input_shape, hidden_size):
    super().__init__()

    self.lstm = LSTM(hidden_size, return_sequences = True, return_state=True, input_shape = [-1, input_shape] )

    self.d1 = Dense(1, input_shape = [-1, hidden_size])

  def call(self, x):

    x, h0, c0 = self.lstm(x)
    x = self.d1(x)

    return x, tf.stack([h0, c0])

model = SimpleModel(input_length, 64)

out, states = model(tf.random.uniform([16,43,256]))

I get the following graph:

3. Failure after conversion
The While OPs fail to run on my target platform unlike the UnidirectionalSequenceLstm OP.
I can only assume that the 'UnidirectionalSequenceLstm' OP doesn't support returning the hidden states? Or am I doing something wrong? My end goal is to feed initial states to the LSTM and retain the returned ones. And I want optimized LSTM Operators. Do I need to write my own conversion logic?

Hi @Doomski99, the UnidirectionalSequenceLSTM op may not allow multiple outputs, the body subgraph of the While op might effectively calculate the same thing as the UnidirectionalSequenceLSTM op. You said this op fails to run on your target platform ... what's your target platform? and what's the error? is there a log or any error message?

Hello and Thank you for your reply @pkgoogle .

the body subgraph of the While op might effectively calculate the same thing as the UnidirectionalSequenceLSTM op

I understand but can you confirm that the performance will remain the same? Could it be that UnidirectionalSequenceLSTM op has a better optimized implementation than a simple While Op?

You said this op fails to run on your target platform ... what's your target platform? and what's the error? is there a log or any error message?

The target platform is Candence's Hifi 5 DSP. I'm using a simulation environment of the latter. For the moment, the model with While runs then stops with a "HALTED" message. Unlike the first model which runs perfectly.

Hello and thank you for your reply @pkgoogle .

the body subgraph of the While op might effectively calculate the same thing as the UnidirectionalSequenceLSTM op

I understand but can you confirm that the performance are the same between the operators? Could it be that the UnidirectionalSequenceLSTM  op has a better optimized implementation?

what's your target platform? and what's the error? is there a log or any error message?

Cadence's HiFi 5 DSP. I'm using a simulation environment. The model with While stops after launching with a "HALTED" message unlike the second model which runs perfectly.

Hi @Doomski99,
I was able to create, load, and run both models on colab: https://colab.sandbox.google.com/gist/pkgoogle/7cd581019a15c5db26d9071e3f968294/62775.ipynb .... do you have any limitations or controls in using your simulated environment? Is there any way to relax those constraints? Just judging from the message and my results, I think that message was produced by your environment rather from TFLite (I would've expected an exception if it was thrown from TFLite)

I understand but can you confirm that the performance is the same between the two operators? Could it be that the UnidirectionalSequenceLSTM op has a better optimized implementation?

You can try (maybe in a general linux system first) to benchmark them: https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/benchmark

Can I get a confirmation?

I can't confirm, here's the code: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/kernels/unidirectional_sequence_lstm.cc It probably depends on what that while body subgraph actually is, i.e. the Op might actually be inside there.

Thank you for your help @pkgoogle !
Before digging through my DSP simulation environment, I've successfully set up the benchmark tool you linked in WSL2 and ran it through my models with the following command:
bazel-bin/tensorflow/lite/tools/benchmark/benchmark_model   --graph=1x_LSTM_hidden_64_float32.tflite   --num_threads=1 --enable_op_profiling=true.
The one with "UnidirectionalSequenceLstm" is consistently 35% faster. Here's the output of both benchmarks:
First (UnidirectionalSequenceLstm) model:
bazel-bin/tensorflow/lite/tools/benchmark/benchmark_model   --graph=1x_LSTM_64_float32.tflite   --num_threads=1 --enable_op_profiling=true
INFO: STARTING!
INFO: Log parameter values verbosely: [0]
INFO: Num threads: [1]
INFO: Graph: [1x_LSTM_64_float32.tflite]
INFO: Enable op profiling: [1]
INFO: #threads used for CPU inference: [1]
INFO: Loaded model 1x_LSTM_64_float32.tflite
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
INFO: The input model file size (MB): 0.330964
INFO: Initialized session in 0.63ms.
INFO: Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
^[[AINFO: count=132 first=3678 curr=3790 min=3270 max=5473 avg=3751.66 std=334

INFO: Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.
INFO: count=261 first=3937 curr=3959 min=3190 max=6333 avg=3790.92 std=358

INFO: Inference timings in us: Init: 630, First inference: 3678, Warmup (avg): 3751.66, Inference (avg): 3790.92
INFO: Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
INFO: Memory footprint delta from the start of the tool (MB): init=5.16016 overall=7.46484
INFO: Profiling Info for Benchmark Initialization:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                         ModifyGraphWithDelegate            0.104           0.104        83.871%         83.871%             0.000              1       ModifyGraphWithDelegate/0
                                 AllocateTensors            0.020           0.020        16.129%        100.000%             0.000              1       AllocateTensors/0

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                         ModifyGraphWithDelegate            0.104           0.104        83.871%         83.871%             0.000              1       ModifyGraphWithDelegate/0
                                 AllocateTensors            0.020           0.020        16.129%        100.000%             0.000              1       AllocateTensors/0

Number of nodes executed: 2
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                         ModifyGraphWithDelegate                1            0.104          83.871%         83.871%          0.000              1
                                 AllocateTensors                1            0.020          16.129%        100.000%          0.000              1

Timings (microseconds): count=1 curr=124
Memory (bytes): count=0
2 nodes observed

INFO: Operator-wise Profiling Info for Regular Benchmark Runs:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                  RANDOM_UNIFORM            0.003           0.003         0.083%          0.083%             0.000              1       [simple_model/random_uniform/RandomUniform]:0
                                  RANDOM_UNIFORM            0.003           0.003         0.070%          0.153%             0.000              1       [simple_model/random_uniform_1/RandomUniform]:1
                    UNIDIRECTIONAL_SEQUENCE_LSTM            3.912           3.769        99.648%         99.801%             0.000              1       [tfl.unidirectional_sequence_lstm]:2
                  Fully Connected (NC, F32) GEMM            0.007           0.008         0.199%        100.000%             0.000              1       Delegate/Fully Connected (NC, F32) GEMM:0

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                    UNIDIRECTIONAL_SEQUENCE_LSTM            3.912           3.769        99.648%         99.648%             0.000              1       [tfl.unidirectional_sequence_lstm]:2
                  Fully Connected (NC, F32) GEMM            0.007           0.008         0.199%         99.847%             0.000              1       Delegate/Fully Connected (NC, F32) GEMM:0
                                  RANDOM_UNIFORM            0.003           0.003         0.083%         99.930%             0.000              1       [simple_model/random_uniform/RandomUniform]:0
                                  RANDOM_UNIFORM            0.003           0.003         0.070%        100.000%             0.000              1       [simple_model/random_uniform_1/RandomUniform]:1

Number of nodes executed: 4
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                    UNIDIRECTIONAL_SEQUENCE_LSTM                1            3.768          99.683%         99.683%          0.000              1
                  Fully Connected (NC, F32) GEMM                1            0.007           0.185%         99.868%          0.000              1
                                  RANDOM_UNIFORM                2            0.005           0.132%        100.000%          0.000              2

Timings (microseconds): count=261 first=3925 curr=3943 min=3186 max=6312 avg=3781.84 std=355
Memory (bytes): count=0
4 nodes observed

Second (While) model:
bazel-bin/tensorflow/lite/tools/benchmark/benchmark_model   --graph=1x_LSTM_hidden_64_float32.tflite   --num_threads=1 --enable_op_profiling=true
INFO: STARTING!
INFO: Log parameter values verbosely: [0]
INFO: Num threads: [1]
INFO: Graph: [1x_LSTM_hidden_64_float32.tflite]
INFO: Enable op profiling: [1]
INFO: #threads used for CPU inference: [1]
INFO: Loaded model 1x_LSTM_hidden_64_float32.tflite
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
WARNING: Attempting to use a delegate that only supports static-sized tensors with a graph that has dynamic-sized tensors (tensor#10 is a dynamic-sized tensor).
INFO: The input model file size (MB): 0.514668
INFO: Initialized session in 0.609ms.
INFO: Running benchmark for at least 1 iterations and at least 0.5 seconds but terminate if exceeding 150 seconds.
INFO: count=107 first=5292 curr=4749 min=4243 max=5357 avg=4657.9 std=246

INFO: Running benchmark for at least 50 iterations and at least 1 seconds but terminate if exceeding 150 seconds.
INFO: Warning: Dropping ProfileBuffer event.
INFO: count=191 first=4983 curr=5046 min=4295 max=6180 avg=4918.73 std=320

INFO: Inference timings in us: Init: 609, First inference: 5292, Warmup (avg): 4657.9, Inference (avg): 4918.73
INFO: Note: as the benchmark tool itself affects memory footprint, the following is only APPROXIMATE to the actual memory footprint of the model at runtime. Take the information at your discretion.
INFO: Memory footprint delta from the start of the tool (MB): init=4.94141 overall=10.2656
INFO: Profiling Info for Benchmark Initialization:
Primary graph profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.018           0.018        19.355%         19.355%             0.000              1       AllocateTensors/0
                         ModifyGraphWithDelegate            0.075           0.075        80.645%        100.000%             0.000              1       ModifyGraphWithDelegate/0

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                         ModifyGraphWithDelegate            0.075           0.075        80.645%         80.645%             0.000              1       ModifyGraphWithDelegate/0
                                 AllocateTensors            0.018           0.018        19.355%        100.000%             0.000              1       AllocateTensors/0

Number of nodes executed: 2
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                         ModifyGraphWithDelegate                1            0.075          80.645%         80.645%          0.000              1
                                 AllocateTensors                1            0.018          19.355%        100.000%          0.000              1

Timings (microseconds): count=1 curr=93
Memory (bytes): count=0
2 nodes observed

Subgraph (index: 1) profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.007           0.007       100.000%        100.000%             0.000              1       AllocateTensors/1

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.007           0.007       100.000%        100.000%             0.000              1       AllocateTensors/1

Number of nodes executed: 1
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                                 AllocateTensors                1            0.007         100.000%        100.000%          0.000              1

Subgraph (index: 1) Timings (microseconds): count=1 curr=7
Memory (bytes): count=0
1 nodes observed

Subgraph (index: 2) profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.025           0.016       100.000%        100.000%             0.000              2       AllocateTensors/2

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.025           0.016       100.000%        100.000%             0.000              2       AllocateTensors/2

Number of nodes executed: 1
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                                 AllocateTensors                1            0.032         100.000%        100.000%          0.000              2

Subgraph (index: 2) Timings (microseconds): count=1 curr=32
Memory (bytes): count=0
1 nodes observed

INFO: Operator-wise Profiling Info for Regular Benchmark Runs:
Primary graph profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                  RANDOM_UNIFORM            0.003           0.003         0.067%          0.067%             0.000              1       [simple_model/random_uniform/RandomUniform]:0
                                       TRANSPOSE            0.027           0.031         0.628%          0.695%             0.000              1       [transpose]:1
                                           WHILE            4.931           4.863        99.305%        100.000%             0.000              1       [while, while1, while2, while3, while4, Unknown]:2

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                           WHILE            4.931           4.863        99.305%         99.305%             0.000              1       [while, while1, while2, while3, while4, Unknown]:2
                                       TRANSPOSE            0.027           0.031         0.628%         99.933%             0.000              1       [transpose]:1
                                  RANDOM_UNIFORM            0.003           0.003         0.067%        100.000%             0.000              1       [simple_model/random_uniform/RandomUniform]:0

Number of nodes executed: 3
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                                           WHILE                1            4.862          99.326%         99.326%          0.000              1
                                       TRANSPOSE                1            0.030           0.613%         99.939%          0.000              1
                                  RANDOM_UNIFORM                1            0.003           0.061%        100.000%          0.000              1

Timings (microseconds): count=191 first=4961 curr=5027 min=4281 max=6154 avg=4896.75 std=315
Memory (bytes): count=0
3 nodes observed

Subgraph (index: 1) profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                            LESS            0.000           0.000        61.756%         61.756%             0.000             37       [simple_model/dense/Tensordot/MatMul1]:0
                                 AllocateTensors            0.003           0.004        38.244%        100.000%             0.000              1       AllocateTensors/1

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.003           0.004        38.244%         38.244%             0.000              1       AllocateTensors/1
                                            LESS            0.000           0.000        61.756%        100.000%             0.000             37       [simple_model/dense/Tensordot/MatMul1]:0

Number of nodes executed: 2
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                                            LESS                1            0.005          62.500%         62.500%          0.000             37
                                 AllocateTensors                1            0.003          37.500%        100.000%          0.000              1

Subgraph (index: 1) Timings (microseconds): count=191 first=9 curr=7 min=3 max=61 avg=9.24084 std=5
Memory (bytes): count=0
2 nodes observed

Subgraph (index: 2) profile:
============================== Run Order ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 AllocateTensors            0.024           0.010         0.263%          0.263%             0.000              1       AllocateTensors/2
                                             ADD            0.001           0.000         0.137%          0.400%             0.000             37       [transpose_1]:0                                 FULLY_CONNECTED            0.018           0.017        16.809%         17.208%             0.000             37       [StatefulPartitionedCall:0]:1
                                          GATHER            0.001           0.001         0.785%         17.993%             0.000             37       [StatefulPartitionedCall:1]:2
                                 FULLY_CONNECTED            0.091           0.068        65.937%         83.930%             0.000             37       [Unknown]:3
                                             ADD            0.001           0.001         0.650%         84.580%             0.000             37       [Unknown]:4
                                             ADD            0.001           0.001         0.623%         85.203%             0.000             37       [Unknown]:5
                                           SPLIT            0.001           0.001         0.639%         85.841%             0.000             37       [Unknown, Unknown, Unknown, Unknown]:6
                                        LOGISTIC            0.001           0.001         0.545%         86.386%             0.000             36       [Unknown]:7
                                        LOGISTIC            0.001           0.000         0.397%         86.783%             0.000             36       [Unknown]:8
                                             MUL            0.001           0.000         0.416%         87.199%             0.000             36       [Unknown]:9
                                        LOGISTIC            0.001           0.000         0.419%         87.618%             0.000             36       [Unknown]:10
                                             ADD            0.000           0.000         0.147%         87.765%             0.000             36       [Unknown]:24
                                   CONCATENATION            0.003           0.004         3.537%         91.302%             0.000             36       [Unknown]:23
                                         RESHAPE            0.000           0.000         0.046%         91.348%             0.000             36       [Unknown]:22
                                           SLICE            0.008           0.004         3.847%         95.195%             0.000             36       [Unknown]:21
                                            TANH            0.000           0.000         0.401%         95.596%             0.000             36       [Unknown]:11
                                             MUL            0.001           0.000         0.265%         95.861%             0.000             36       [Unknown]:12
                                             ADD            0.000           0.000         0.221%         96.082%             0.000             36       [Unknown]:13
                                            TANH            0.000           0.000         0.360%         96.442%             0.000             36       [Unknown]:14
                                             MUL            0.001           0.000         0.261%         96.703%             0.000             36       [Unknown]:15
                                         RESHAPE            0.000           0.000         0.059%         96.762%             0.000             36       [Unknown]:16
                                   CONCATENATION            0.001           0.000         0.282%         97.044%             0.000             36       [Unknown]:17
                                           SLICE            0.001           0.003         2.774%         99.818%             0.000             36       [Unknown]:18
                                         RESHAPE            0.000           0.000         0.049%         99.867%             0.000             36       [Unknown]:19
                                   CONCATENATION            0.000           0.000         0.133%        100.000%             0.000             36       [Unknown]:20

============================== Top by Computation Time ==============================
                                     [node type]          [first]        [avg ms]            [%]          [cdf%]          [mem KB]      [times called]  [Name]
                                 FULLY_CONNECTED            0.091           0.068        65.937%         65.937%             0.000             37       [Unknown]:3
                                 FULLY_CONNECTED            0.018           0.017        16.809%         82.745%             0.000             37       [StatefulPartitionedCall:0]:1
                                 AllocateTensors            0.024           0.010         0.263%         83.008%             0.000              1       AllocateTensors/2
                                           SLICE            0.008           0.004         3.847%         86.855%             0.000             36       [Unknown]:21
                                   CONCATENATION            0.003           0.004         3.537%         90.393%             0.000             36       [Unknown]:23
                                           SLICE            0.001           0.003         2.774%         93.167%             0.000             36       [Unknown]:18
                                          GATHER            0.001           0.001         0.785%         93.951%             0.000             37       [StatefulPartitionedCall:1]:2
                                             ADD            0.001           0.001         0.650%         94.601%             0.000             37       [Unknown]:4
                                           SPLIT            0.001           0.001         0.639%         95.240%             0.000             37       [Unknown, Unknown, Unknown, Unknown]:6
                                             ADD            0.001           0.001         0.623%         95.863%             0.000             37       [Unknown]:5

Number of nodes executed: 26
============================== Summary by node type ==============================
                                     [Node type]          [count]         [avg ms]          [avg %]         [cdf %]       [mem KB]      [times called]
                                 FULLY_CONNECTED                2            3.157          83.035%         83.035%          0.000             74
                                           SLICE                2            0.251           6.602%         89.637%          0.000             72
                                   CONCATENATION                3            0.150           3.945%         93.582%          0.000            108
                                             ADD                5            0.065           1.710%         95.292%          0.000            183
                                        LOGISTIC                3            0.050           1.315%         96.607%          0.000            108
                                             MUL                3            0.034           0.894%         97.501%          0.000            108
                                          GATHER                1            0.029           0.763%         98.264%          0.000             37
                                            TANH                2            0.028           0.736%         99.001%          0.000             72
                                           SPLIT                1            0.024           0.631%         99.632%          0.000             37
                                 AllocateTensors                1            0.010           0.263%         99.895%          0.000              1
                                         RESHAPE                3            0.004           0.105%        100.000%          0.000            108

Subgraph (index: 2) Timings (microseconds): count=191 first=3863 curr=3956 min=3340 max=4844 avg=3816.78 std=256
Memory (bytes): count=0
26 nodes observed

It's interesting that we can see the operations happening inside the While loop through this tool.

It probably depends on what that while body subgraph actually is, i.e. the Op might actually be inside there.

I guess the Op isn't there after all. It seems that it's splitting the LSTM operation.  What I find interesting is that, in the second model, the "subgraph index = 2" that is mentioned inside the While loop is averaging at 3.8ms, which is close to that of the UnidirectionalSequenceLSTM, but  the actual while loop is averaging at 4.8ms, more than the "subgraph index = 2". Not sure if that's relevant or not.
Anyways, I've also tried another LSTM model of mine that is dynamically quantized, and I'm getting a 40% speedup. I don't think we can justify the speedup for simply not retaining the hidden_states, right?
Performance is critical for my task, how do you think we should proceed? I'll be waiting for your input.

So the converter is in some sense an arbitrary program optimizer, since TF is Turing complete, it probably doesn't handle all cases to complete optimum, there probably exists a conversion which is more optimal (or maybe we need to change the source code/that op to more output those hidden states natively rather than the deconstruction/reconstruction you saw). This will likely be a hard problem to solve generally and you say it's critical for your application. We will get to it eventually but we have to consider all issues and how many issues/users each potential change can help with and their relative impact as well -- so it might take a while to get to this one. That being said, since it is impactful for you and now that you are touching the innards a bit you can try your best to see if you can introduce a PR for this yourself.
Let's change this into a feature/performance issue and I can direct you a little bit on what to look at but you'll likely have to do a lot of digging for yourself. Learn some MLIR https://mlir.llvm.org/ (A very general compiler framework ... a compiler translates higher level code to lower level code, but some of the same abstractions can be used to .. side translate/optimize such as TF -> TFL, optimization can be viewed as lowering to a representation more efficient for your specific configuration/hardware)
Build from source with debug info, especially the tflite_convert CLT: https://www.tensorflow.org/lite/models/convert/convert_models#command_line_tool, use lldb/gdb with it and dig into how your program gets transformed and see how we can do the optimization better
This README is probably useful: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/README.md
Some of the binaries here might be useful:
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/compiler/mlir/lite/BUILD
Hi @majiddadashi, can you please take a look? Thanks.

Hi @Doomski99, can you try it this way and see if it satisfies your performance requirements?
import torch
from torch import nn
import ai_edge_torch

class SimpleModel(nn.Module):
    def __init__(self, input_size, hidden_size):
        super().__init__()

        self.lstm = nn.LSTM(input_size, hidden_size)
        self.d1 = nn.Linear(hidden_size, 1)

    def forward(self, x):

        x, (h0, c0) = self.lstm(x)
        x = self.d1(x)

        return x, torch.stack([h0, c0])

model = SimpleModel(256, 64)
sample_inputs = (torch.randn(16, 43, 256),)

edge_model = ai_edge_torch.convert(model.eval(), sample_inputs)
edge_model.export("simple_model.tflite")

You can find more information here: AI-Edge-Torch and googleblog

Hello @pkgoogle . Interesting library, thanks for sharing! I tested it and it seems to unroll the LSTM instead of using a loop (while) which is not quite what I was looking for. My goal was to be able to have the "UnidirectionalSequenceLSTM" Operator instead.

Hi @Doomski99, thanks for trying! I think this is a great datapoint for that team... would you mind creating an issue there?

Hi @Doomski99, thanks for trying! I think this is a great datapoint for that team... would you mind creating an issue there?

Sure! I submitted it here.

Hi, @Doomski99
Thanks for raising this issue. Are you aware of the migration to LiteRT? This transition is aimed at enhancing our project's capabilities and providing improved support and focus for our users. As we believe this issue is still relevant to LiteRT we are moving your issue there. Please follow progress here: https://github.com/google-ai-edge/LiteRT/issues/87
Let us know if you have any questions. Thanks.


[Enhancement] Incremental Encoder Example

As you provided a simple example of the SSI Encoder using VHDL, it would be nice to get the Incremental Encoder example.
The type of Encoder is also very commun for Automation Industry users.
The example should correspond to  standard linear or rotary devices https://en.wikipedia.org/wiki/Incremental_encoder

The Incremental Encoder example is now available to download.


Path with arrow do not update properly when moved

Hi,
This is Gabriel.
Thanks for the nice library.
I have a question about path which has arrow.
This is working with FF, Chrome and IE8, IE9.
As you can see when you move the rectangle the linked path's arrow do not move with arrow in IE10.
http://jsfiddle.net/TnhVn/7/
Please check this.
Best regards,
Gabriel.

I was able to get around this issue although my solution may not work for you since your implementation is far more dynamic.
My solution was forcing the container element around the svg canvas to repaint the entire canvas by reappending it.
For example:
svgContainer.appendChild(svgContainer.childNodes[0]);
For my case, the redraw isn't too intensive since it only happens once after a specific action.


Created navbar sticky and set correct redirection links of navbar

Pull Request Checklist
Fixes #1120
screenshot:

[X] I have added screenshots and videos to show before and after the working of my code.
[X] I have ensured that the screen size is set to 100% while making the video.
[X] I have synced the latest fork with my local repository and resolved any conflicts.
[X] I have mentioned the issue number which I created before making this PR .(format to mention issue number is : fixes #(issue number) )
[X] I understand that if any the above conditions are not met , my PR will not be MERGED .

hey @ANSHIKA-26  check this out is there any problem?

can you record a video of the changes done ?

https://github.com/user-attachments/assets/b3564f59-cd4f-42ca-8909-cb6f0d8db1d2
github not supporting large videos

I have also corrected redirection links in navbar


CentOS 6 Glibc Fix not working for Garrys mod

It appears that Glibc fix is not working for Garrys mod. I will be addressing this issue.

8a6d7b92b739ff5826a5320d5f3d29a496c3b352


Correction for raw affiliation Food and Healthy Living, Aeres University of Applied Sciences Almere, Stadhuisstraat 18, Almere, 1315 HC, the Netherlands; Urban Economics, Department of Social Sciences, Wageningen University, Hollandseweg 1, 6706 KN Wageningen, the Netherlands

Correction needed for raw affiliation Food and Healthy Living, Aeres University of Applied Sciences Almere, Stadhuisstraat 18, Almere, 1315 HC, the Netherlands; Urban Economics, Department of Social Sciences, Wageningen University, Hollandseweg 1, 6706 KN Wageningen, the Netherlands
raw_affiliation_name: Food and Healthy Living, Aeres University of Applied Sciences Almere, Stadhuisstraat 18, Almere, 1315 HC, the Netherlands; Urban Economics, Department of Social Sciences, Wageningen University, Hollandseweg 1, 6706 KN Wageningen, the Netherlands
new_rors: 03jnx2c74;04qw24q55
previous_rors: 04qw24q55
works_examples: W4206387082
contact: 614766ebda44d8985109f95e0d4837fe:fc286c49f76c06f809d6 @ hotmail.com

This issue was accepted and ingested by the OpenAlex team on 2024-10-10. The new affiliations should be visible within the next 7 days.


Proper rewrites

** Please describe which feature you have a question about? **
Rewrites and Redirects
** Provide additional details**
I am trying to configure a domain to host both the client and API behind it. The application is a URL Shortener. Here is an example of what I am trying to do using use cases.

User hits https://mydomain.com -> directed to the amplify console application (including all resource files like JS, CSS, etc)
User hits https://mydomain.com/shortcut -> rewrites to cloudfront domain.

If shortcut exists -> 301 redirect to target domain.
If does not exist -> 301 redirect back to https://mydomain.com?error=url_not_found

So here is some more info. The following works but seems more than needed.
[
    {
        "source": "https://www.slip.link",
        "target": "https://slip.link",
        "status": "302",
        "condition": null
    },
    {
        "source": "/",
        "target": "/index.html",
        "status": "200",
        "condition": null
    },
    {
        "source": "/css/<*>",
        "target": "/css/<*>",
        "status": "200",
        "condition": null
    },
    {
        "source": "/js/<*>",
        "target": "/js/<*>",
        "status": "200",
        "condition": null
    },
    {
        "source": "/favicon.ico",
        "target": "/favicon.ico",
        "status": "200",
        "condition": null
    },
    {
        "source": "/<*>",
        "target": "https://d16140obdnjb2p.cloudfront.net/<*>",
        "status": "200",
        "condition": null
    }
]

I feel like I could reduce this with soimething like...
</^(?!(\/css\/|\/js\/|\/index.html|favicon.ico|\/\?)).+/>

@singledigit  can you try this?
<^(?!(css|js|index.html|favicon.ico|/?)).+>


Use ByteArray to represent strings

Most notably in ERC20, ERC721, and ERC1155.

⚠ Warning: Still not supported in Starknet.

When will this feature be completed?

We expect this to be merged within 1 to 3 weeks

We expect this to be merged within 1 to 3 weeks, what do you need it for? What are you building?

Thank you, I will wait to use it.


Dev measure aggregate function

Add several functions

[ ] If this is non-trivial feature, paste the links/URLs to the design doc.
[ ] Update the documentation to include this new feature.
[x] Tests(including UT, IT, E2E) are added to verify the new feature.
[ ] If it's UI related, attach the screenshots below.

Here are new added functions:

max
count
sum
percent
rate

Please rebases the commits to avoid so long commit logs.

Benchmark loop-unrolling
The benchmark code is here.
On my machine, the result is:

We can see that, with the round increasing, the efficiency difference is decreasing.


Revert "Copy visualizations using export/import"

Reverts CartoDB/cartodb#8045

In Github we trust!
📦 it
I'd wait for 🍏 just in case. We're not in that much of a hurry.


Fixed multiline scenario bug and warning

This fixes the issue  Feature/Scenario description can only be 1 line

Haha wow, that's one intense rule!

Haha wow, that's one intense rule!

Published in v0.8.4 :)

Published in v0.8.4 :)

@incident-recipient @bbqsrc I'm not sure why, but it looks like this PR somehow broke cucumber-rust tests :man_shrugging:

@incident-recipient @bbqsrc I'm not sure why, but it looks like this PR somehow broke cucumber-rust tests :man_shrugging:

Not ideal this.

Not ideal this.

@ilslv I've yanked 0.8.4. Issue is easily reproducable. Hopefully will have a solution this evening.

@ilslv I've yanked 0.8.4. Issue is easily reproducable. Hopefully will have a solution this evening.


[SPARK-40715][SQL] Support selecting shuffled hash join thought LocalMapThreshold is less than advisory partition size

What changes were proposed in this pull request?
Now preferring shuffled hash join condition:

ADVISORY_PARTITION_SIZE_IN_BYTES <= ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD
All partitions size < ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD
but ADVISORY_PARTITION_SIZE_IN_BYTES is 64MB by default, so we can not add SHUFFLE_HASH hint for the plan which all partitions size are less than ADVISORY_PARTITION_SIZE_IN_BYTES.

The new preferring shuffled hash join condition:

Try to coalesce the join, thinking the side satisfies Shuffled Hash Join condition if all the coalesced partitions are less than ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD

Why are the changes needed?
Change more join to shuffled hash join.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Exists UT

Can one of the admins verify this patch?

CoalesceShufflePartitions will make each partition size close to ADVISORY_PARTITION_SIZE_IN_BYTES unless some partitions are skewed. So it has no meaning to compare with AUTO_BROADCASTJOIN_THRESHOLD...
I'm thinking to add a new rule to optimize smj to shj by spliting the bigger partition to smaller (like skew join did), then we can build hash relation safely. But it only works for inner join, since we need split build side.

Hi, @ulysses-you Thanks for your review.

For example, if ADVISORY_PARTITION_SIZE_IN_BYTES is 64M, the two sides of join are : [30M, 30M, 30M, 30M, 30M] and [100M, 100M, 100M, 100M, 100M], CoalesceShufflePartitions will transform ShuffleQueryStageExec to  CustomShuffleReaderExec(stage, ShufflePartitionSpec[130M, 130M, 130M, 130M, 130M]) , this query should be able to convert to ShuffledHashJoin (not support now).
If we enable BROADCASTJOIN,  I think the data that will be loaded into memory and build HashRelation should also less than AUTO_BROADCASTJOIN_THRESHOLD,  i.e. ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD <= AUTO_BROADCASTJOIN_THRESHOLD

For example, if ADVISORY_PARTITION_SIZE_IN_BYTES is 64M, the two sides of join are : [30M, 30M, 30M, 30M, 30M] and [100M, 100M, 100M, 100M, 100M], CoalesceShufflePartitions will transform ShuffleQueryStageExec to CustomShuffleReaderExec(stage, ShufflePartitionSpec[130M, 130M, 130M, 130M, 130M]) , this query should be able to convert to ShuffledHashJoin (not support now).

I'm not sure what you mean, join has two shuffle stages so should also have two shuffle reads. For your example, CoalesceShufflePartitions won't coalesce partition since the sum of two continuous partition size bigger than 64MB.  You can find some details in test ShufflePartitionsUtilSuite. And this example has already supported to convert to shj if you set ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD to 64MB.

If we enable BROADCASTJOIN, I think the data that will be loaded into memory and build HashRelation should also less than AUTO_BROADCASTJOIN_THRESHOLD, i.e. ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD <= AUTO_BROADCASTJOIN_THRESHOLD

As I mentioned, even if all partition size less than AUTO_BROADCASTJOIN_THRESHOLD, the final partition size of shuffle reads which are optimized by CoalesceShufflePartitions will close to ADVISORY_PARTITION_SIZE_IN_BYTES. Then it breaks your assumption.

For example, if ADVISORY_PARTITION_SIZE_IN_BYTES is 64M, the two sides of join are : [30M, 30M, 30M, 30M, 30M] and [100M, 100M, 100M, 100M, 100M], CoalesceShufflePartitions will transform ShuffleQueryStageExec to CustomShuffleReaderExec(stage, ShufflePartitionSpec[130M, 130M, 130M, 130M, 130M]) , this query should be able to convert to ShuffledHashJoin (not support now).

I'm not sure what you mean, join has two shuffle stages so should also have two shuffle reads. For your example, CoalesceShufflePartitions won't coalesce partition since the sum of two continuous partition size bigger than 64MB. You can find some details in test ShufflePartitionsUtilSuite. And this example has already supported to convert to shj if you set ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD to 64MB.

It can not be converted to shj If ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD is less than 64MB, eg ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD = 40M?
Our production executor has 18 vCores, so we want to set ADAPTIVE_MAX_SHUFFLE_HASH_JOIN_LOCAL_MAP_THRESHOLD to a small value.

Hi, @ulysses-you  could you help to review this PR again? Thanks