[AnimatedVisualPlayer]: Fix AnimatedVisualPlayer should work on RS2+ (and Fallback only if Source fails) (#363)

This change is to fix AnimatedVisualPlayer should work on RS2+ lower than RS5 with Fallback only if Fallback only. This regression was caused by an accidentally adding OS check in method AnimatedVisualPlayer::UpdateContent() because we would like to fix a crash from c# Lottie source which does not have OS check because some reason. This attempting fix caused such regression. Thus, the fix is to remove such check in ::UpdateContent() while keeping OS check in C# lottie source.

Will need to coordinate with @ranjeshj 's checkin for updating the test projects to not use project.json.


[1.9] Add enhancement to Soul Matter Hammer

onItemUse is overridden so make it so if we shift-click a special block we get the item and if not return super.onItemUse

Will be added in v2.1.1


Improve widgets error state for supporting several error types

When there is an error in the Dataviews request, widgets can be rendered. We display an error saying "There has been an error" (or something like that), and a refresh button (is it useful?).
From now on we should be more concrete about the problem. We should catalog those by:

Data error: any dataview request returned an error and we should display it.
Limits error: the user has reached its platform limits, dataview request will fail but with a specific error code or message, so we should decide if the error message should change or not (let's not forget this will be public in the embed page).

Design

*Related ticket: https://github.com/CartoDB/cartodb-platform/issues/3279

Hi! I've seen in this issue https://github.com/CartoDB/cartodb/issues/12751 the widget error text has changed, right now, for the limits engine is sending this error:
{
  type: 'limit',
  message: 'You are over platform\'s limits. Please contact us to know more details',
  subtype: 'datasource'
}

Should we separate both errors, timeout (poor connectivity, error with the server) and limits (database/tiler timeout), or keep the same error for both of them?
Also, I've realised, since we're going to have more feedback about the errors, is it still necessary the design with only one button or can we use the last one in this design for that case?

cc @CartoDB/design

IMO, we should separate them. We might want to improve the copy for that limits error and prepare the visual.
Regarding your last question: if we have specific information about the error (like limits or poor connectivity) I would show the full message to properly explain it. But if we don't, I recommend avoiding the repetition of elements. Right now, in the following case, the buttons already feel super repetitive, if we also add a generic message and repeat it all over, it will be too much.

I totally agree, @CartoDB/design . New ticket for that, please!

I think we already have both tickets, this one for the errors and the other one for the button styles, right?
Btw, right now when there is no data, the widget shows emtpy:

Should I replace that with the "No data available" design?

Yes and yes @rubenmoya

On production 🚢

:dancer:


sap.m.FeedListItem: icon related settings not updatable after initial rendering

OpenUI5 version:
Since https://github.com/SAP/openui5/commit/454a41c8b7034c673a9c34b92921ced54b7bdce5#diff-8b72ed9ca72d9bf06081308bacffad14f5719bade70ed486635cd5ec2bfabdfa
Steps to reproduce the problem

Open https://jsbin.com/megaguq/edit?html,js,output
Using the Control Tree diagnostic tool, play around with icon related settings such as setting icon to sap-icon://sap-ui5.

We can see that those changes are not applied even though the FeedListItem itself does get invalidated and rerenders.
Because of this issue, also activeIcon fails to work since 1.88.
The main issue is at _getAvatar:
https://github.com/SAP/openui5/blob/98bb2bda851303610df5809ae251adb7712049cc/src/sap.m/src/sap/m/FeedListItem.js#L441-L456
Unlike FeedInput, subsequent changes on Avatar are never applied.
A quick fix could look something like this:
FeedListItem.prototype._getAvatar = function() {
	var sIcon = this.getIcon();
	var sIconSrc = sIcon ? sIcon : "..."; // another issue: https://github.com/SAP/openui5/issues/3258
	var sId = this.getId() + "-icon";

	this.oAvatar = this.oAvatar || new Avatar(sId);
	this.oAvatar.applySettings({
		src: sIconSrc,
		displayShape: this.getIconDisplayShape(),
		initials: this.getIconInitials(),
		displaySize: this.getIconSize(),
		ariaLabelledBy: this.getSender()
	});
	// ...

But while we're at it: Avatar is currently defined with var Avatar = library.Avatar. Wouldn't it be better to properly require it in the dependency list early and then using "sap.m.Avatar" as the type of a hidden aggregation? Similar to _avatar in ListContentItem or _placeholder in ObjectPageHeaderContent. FeedListItems as well as FeedInputs are typically used with profile pictures as shown in the design guideline. Or what are the reasons for declaring the dependency like this var Avatar = library.Avatar?

Changes is done for this issue in the below gerrit link:
https://git.wdf.sap.corp/c/openui5/+/5292634

Hello,
As the issue seem to be fixed, I am closing the issue.
Best regards,
Tereza


Added new resource(An explanation of CC)

Fixes

Fixes #86 by @nown1ne
Description
Added new/latest resource which consist of informative slide deck covering everything you need to know about copyright and Creative Commons.
Screenshots

Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main or master).
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[ ] I added or updated tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no
visible errors.

Developer Certificate of Origin

For the purposes of this DCO, "license" is equivalent to "license or public domain dedication," and "open source license" is equivalent to "open content license or public domain dedication."

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Hi! @possumbilities @TimidRobot  Please have a look and review my PR and advise if any changes are needed? Your guidance is strongly required.
Thanks

Hi @praveen-rikhari thanks for this contribution, you need to remove the change you committed for Gemlock.file.

Thanks @possumbilities for your consideration.Yes working on it

Hey @possumbilities I commited the changes. Please review and let me know if any other changes required.
Thanks

@praveen-rikhari If you comment on the original Issue I'll Assign you there.


Large suite of changes - Including removing dependency on Material.Icons

Major Changes:
Completely removed dependency on Material.Icons/Material.Icons.Avalonia.

This prevents issues where installing a different version of the package into a project would cause icons in SukiUI to be wrong.
Completely unnecessary dependency anyway, a few mb for what turned out to be 30 icons.
New solution involves a simple C# file with icon paths stored in StreamGeometry instances, displayed with PathIcon
Now allows custom icons to be included or icons from other sources to be used without requiring another dependency.
Provides compile time safety, and allows icons to be changed in one place and the change being consistent across the whole UI.
Allows devs to reference these named icons themselves to improve consistency in their apps, which will also be immune to updates/changes to SukiUI.
Unfortunately sizing these PathIcons is different so I simply stripped out all sizes so they can be adjusted in the future.

Enabled nullables for the project

This is the standard now generally and can prevent significant issues going forward, this can be unwound if it's seen to be an issue.

Minor Changes:

WaveProgress converters rewritten to prevent try/catch abuse that can drain performance.
Simplified Maximise/Restore window button logic, icon is now simply set by a style.
Added optional BusyText property to the BusyArea which indicates to the user what the app is doing.
Enabled the sort icon on DataGrid by giviing it a color and changed enable/disable behaviour to use Opacity so that space is correctly reserved for it, preventing jerky resizing behaviour.

Fixes:

Include .idea/ in the .gitignore for Rider devs.
Stripped out all the margins/padding present in the Button styles as these seemed to do nothing but cause issues.
Title bar interaction now spans it's entire height.
Implemented a style applied on Windows only that applies an 8px padding to the window to offset an issue in Windows where windows that are maximised are over-sized to hide the system borders, but SukiUI's NakedWindow removes those system borders.
Removed duplicate image files causing issues developing on Windows (Fixes #55)

Outstanding Issues:

As stated before, when replacing all the MaterialIcon controls, the Height and Width properties simply had to be removed because the sizing wasn't consistent, going forward they may have to be adjusted.
Buttons haven't been fully tested after the major changes to the padding/margins, however they seem okay.
If users are using Material.Icons themselves, these icons won't have their foreground set correctly by default as before, however devs can do this with a single simple style anyway.

Okay, this is really cool ! 🚀
Thank you for your contribution, these are exactly the kind of improvements the library needs.


Retire legacy code in torchtext

❓ Questions and Help
As the maintainer of the torchtext library, we plan to retire torchtext.data.batch.Batch and torchtext.data.example.Example in the next release (by the end of October). Those two components will be still available in torchtext.legacy. To handle the compatibility issue, should we send a PR and fix this on GitHub first, or we could just make some changes in fbcode?
Here are the proposed changes:
# old API
from torchtext.data.batch import Batch
from torchtext.data.example import Example

# new API
from torchtext.legacy.data.batch import Batch
from torchtext.legacy.data.example import Example

Relevant issue: https://github.com/pytorch/text/issues/985

I think when you know from which version you are going to change it we can implement it right away on one of the following ways (also if you have another proposal it is welcome)

with known version:if torchtext.__version__ < X.Y:
    from torchtext.data.batch import Batch
    from torchtext.data.example import Example
else:
    from torchtext.legacy.data.batch import Batch
    from torchtext.legacy.data.example import Example

generic ways:try:
    from torchtext.legacy.data.batch import Batch
    from torchtext.legacy.data.example import Example
except ImportError:
    from torchtext.data.batch import Batch
    from torchtext.data.example import Example

I think when you know from which version you are going to change it we can implement it right away on one of the following ways (also if you have another proposal it is welcome)

with known version:if torchtext.__version__ < X.Y:
    from torchtext.data.batch import Batch
    from torchtext.data.example import Example
else:
    from torchtext.legacy.data.batch import Batch
    from torchtext.legacy.data.example import Example

generic ways:try:
    from torchtext.legacy.data.batch import Batch
    from torchtext.legacy.data.example import Example
except ImportError:
    from torchtext.data.batch import Batch
    from torchtext.data.example import Example

Thanks. We will know which the version we will change. I'm just wondering if I should change this on Github first, or I can send a Diff internally to fbcode and later on sync with the master branch. I just want to know the mechanism to sync PyTorchLightening between fbcode and github.

you can prepare a PR and later ping us when it actually, just not to forget it...
even throw we would find it as soon as you release it that some our tests are failing :]

Sounds good. For the timeline, we will drop those legacy code from the master branch early October and eventually in 0.8.0 release (10/27/2020).

so I think that if you decided to go with version check you can send it now...

Thanks. We will know which the version we will change. I'm just wondering if I should change this on Github first, or I can send a Diff internally to fbcode and later on sync with the master branch. I just want to know the mechanism to sync PyTorchLightening between fbcode and github.

@zhangguanheng66 There is none. PyTorch Lightning isn't linked to internal pytorch

Thanks. We will know which the version we will change. I'm just wondering if I should change this on Github first, or I can send a Diff internally to fbcode and later on sync with the master branch. I just want to know the mechanism to sync PyTorchLightening between fbcode and github.

@zhangguanheng66 There is none. PyTorch Lightning isn't linked to internal pytorch

@ananthsub Thanks. So I have to manually fix the internal version by myself, right? Is there anyone importing OSS PyTorch Lightning to fbcode periodically?

That’s me + our team. We have scripts to do this

@zhangguanheng66 what is the progress here?

@zhangguanheng66 what is the progress here?

We are still cleaning up the new building blocks in torchtext so we postponed the retirement of the legacy code (maybe Nov. and December).

@zhangguanheng66 what is the progress here?

We are still cleaning up the new building blocks in torchtext so we postponed the retirement of the legacy code (maybe Nov. and December).

@zhangguanheng66 any update here, the freeze for <0.7 becoming a blocker for us for an upgrade on python 3.9 as the first torch supporting  it is torch>=1.7.1 and so your torchtext==0.8.1

For now, you can use 0.8.1 for torchtext, although the legacy code is still there.

We have landed the PR to retire the legacy code https://github.com/pytorch/text/pull/1172. This will be released in torchtext v0.9.0, and we are targeting the release date on March 4.
If you still prefer to use batch, you can add the legacy tag to the path, a.k.a.
from torchtext.legacy.data.batch import Batch

we have some legacy separation in https://github.com/PyTorchLightning/pytorch-lightning/pull/6211/files


need help on setting up lavalink in aws

sir i downloaded the letest lavalink file and my application.yml looks like this
server: # REST and WS server
  port: 2333 (if i use both address below then i get unable to bind the port but if i use local host how can i access to my discord bot )
  address: <IP_ADDRESS>                  //<IP_ADDRESS> (public ip address ) <IP_ADDRESS> (private ip address ) cannot bind to these ip address gives me error
                                           // If i use here  <IP_ADDRESS>  (working here but how can i access it outside this es2 server?)
lavalink:
  server:
    password: "youshallnotpass"
    sources:
      youtube: true
      bandcamp: true
      soundcloud: true
      twitch: true
      vimeo: true
      mixer: true
      http: true
      local: false
    bufferDurationMs: 400
    youtubePlaylistLoadLimit: 6 # Number of pages at 100 each
    youtubeSearchEnabled: true
    soundcloudSearchEnabled: true
    gc-warnings: true

metrics:
  prometheus:
    enabled: false
    endpoint: /metrics

sentry:
  dsn: ""
#  tags:
#    some_key: some_value
#    another_key: another_value

logging:
  file:
    max-history: 30
    max-size: 1GB
  path: ./logs/

  level:
    root: INFO
    lavalink: INFO

how can i set this in aws so that i can access the lavalink via public ip (added a secondary ip address which is given above)
it seems i opened all the ports for time being still having issue
this is my output
2020-02-07 05:19:44.083  INFO 17127 --- [           main] lavalink.server.Launcher                 :

       .   _                  _ _       _    __ _ _
      /\\ | | __ ___   ____ _| (_)_ __ | | __\ \ \ \
     ( ( )| |/ _` \ \ / / _` | | | '_ \| |/ / \ \ \ \
      \\/ | | (_| |\ V / (_| | | | | | |   <   ) ) ) )
       '  |_|\__,_| \_/ \__,_|_|_|_| |_|_|\_\ / / / /
    =========================================/_/_/_/

        Version:        8d2dabb19ff31d254e5778997ef1244d73e4a8b9-SNAPSHOT
        Build:          943
        Build time:     28.11.2019 01:35:07 UTC
        Branch          master
        Commit:         8d2dabb
        Commit time:    28.11.2019 01:33:04 UTC
        JVM:            1.8.0_242
        Lavaplayer      1.3.22

2020-02-07 05:19:44.322  INFO 17127 --- [           main] lavalink.server.Launcher                 : Starting Launcher on ip-172-31-31-46 with PID 17127 (/home/bitnami/uploads/Lavalink.jar started by bitnami in /home/bitnami/uploads)
2020-02-07 05:19:44.324  INFO 17127 --- [           main] lavalink.server.Launcher                 : No active profile set, falling back to default profiles: default
2020-02-07 05:19:44.636  WARN 17127 --- [kground-preinit] o.s.h.c.j.Jackson2ObjectMapperBuilder    : For Jackson Kotlin classes support please add "com.fasterxml.jackson.module:jackson-module-kotlin" to the classpath
2020-02-07 05:19:46.329  WARN 17127 --- [           main] io.undertow.websockets.jsr               : UT026010: Buffer pool was not set on WebSocketDeploymentInfo, the default pool will be used
2020-02-07 05:19:46.362  INFO 17127 --- [           main] io.undertow.servlet                      : Initializing Spring embedded WebApplicationContext
2020-02-07 05:19:46.362  INFO 17127 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 1897 ms
2020-02-07 05:19:46.502  WARN 17127 --- [           main] l.server.config.SentryConfiguration      : Turning off sentry
2020-02-07 05:19:46.509  INFO 17127 --- [           main] l.s.c.AudioSendFactoryConfiguration      : OS: Linux, Arch: amd64
2020-02-07 05:19:46.509  INFO 17127 --- [           main] l.s.c.AudioSendFactoryConfiguration      : JDA-NAS supported system detected. Enabled native audio sending.
2020-02-07 05:19:46.509  INFO 17127 --- [           main] l.s.c.AudioSendFactoryConfiguration      : Setting buffer to 400ms
2020-02-07 05:19:46.510  INFO 17127 --- [           main] l.s.c.AudioSendFactoryConfiguration      : Setting playlist load limit to 6
2020-02-07 05:19:47.342  INFO 17127 --- [           main] c.s.d.l.tools.GarbageCollectionMonitor   : GC monitoring enabled, reporting results every 2 minutes.
2020-02-07 05:19:47.910  INFO 17127 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2020-02-07 05:19:48.407  INFO 17127 --- [           main] org.xnio                                 : XNIO version 3.3.8.Final
2020-02-07 05:19:48.426  INFO 17127 --- [           main] org.xnio.nio                             : XNIO NIO Implementation Version 3.3.8.Final
2020-02-07 05:19:48.533  INFO 17127 --- [           main] o.s.b.w.e.u.UndertowServletWebServer     : Undertow started on port(s) 2333 (http) with context path ''
2020-02-07 05:19:48.537  INFO 17127 --- [           main] lavalink.server.Launcher                 : Started Launcher in 5.411 seconds (JVM running for 6.243)

how can i access it to my pc

Using <IP_ADDRESS> will bind to all IPs.

how can i access it to my pc

Depends on your client. I am not responsible for that.

will hosting lavalink on private ip show on public ip as its linked ?

No.
Think of private IP as LAN.
This issue tracker is not for general troubleshooting. Please discuss the problem in our Discord channel instead.


cbhwlib/XmlFile.cpp fails to compile since changes for Qt6 compatiblity in commit 44d8ebfd6e6567c1aafde279ca5c673d847efc01

I don't know if it is considered possible to have the code compile with qt5 and qt6 at the same time, bit I would very much appreciate a transitional period which would allow to use both, qt5 and qt6, at the same time.

As mentioned in the linked PR, any efforts to fix Qt compatibility should go towards ripping out the Qt dependency altogether, so I'm closing this issue.
All that remains of Qt is XML, a QApplication (which can be replaced with a main and some threads), and a couple signals and slots. I'm not sure how to replace the latter, but here's a discussion on signaling libraries in C. It looks like asio/signals2 is a popular combination, which could also be applied to the QAapplication replacement.
I looked into replacing QXML with pugixml and it's not so bad. I just need to decide if I want to do a functional-only replacement or if I want to create something like a libCCF.


[Protection] Add PopCap DRM Protect

Inhouse copy protection used by shareware PopCap games, seemingly replacing the previous use of ActiveMARK. Seems to make use of a drm.xml found in various places depending on the time the game was released, with earlier versions placing drm.xml in the folder "drm/common", later versions in the root directory of the game. Later versions use a seemingly obfuscated drm.xml.bin, with every version of the xml/bin having a corresponding .sig file. The "main" exe is actually just a wrapper for the actual executable, seemingly stored in a DAT file with the same name. When the wrapper is run, it extracts the real executable as a hidden file in the working directory with the name popcap1.exe (not sure if there are others). Strings worth investigating: "?popcapdrmprotect!", "?popcapdrmprotend!", and "!YN00000PACPOPPOPCAPPACPOPPOPCAPBUILDINFOMARKERPACPOPPOPCAPPACPOPPOPCAPXXXXXXXXX". Files that may worth doing checks for: "drm.xml", "drm.xml.sig", "drm.xml.bin", "drm.xml.bin.sig", "drmss.jpg".

Seems to also have a CD equivalent, the following disc appears to have it: http://redump.org/disc/59954
Files to investigate:


[AIRFLOW-3090] Demote dag start/stop log messages to debug

Make sure you have checked all steps below.
Jira

[ x] My PR addresses the following Airflow Jira issues and references them in the PR title. For example, "[AIRFLOW-XXX] My Airflow PR"

https://issues.apache.org/jira/browse/AIRFLOW-3090
In case you are fixing a typo in the documentation you can prepend your commit with [AIRFLOW-XXX], code changes always need a Jira issue.

Description

[ x] Here are some details about my PR, including screenshots of any UI changes:

Demote two log messages from info to debug
Tests

[ x] My PR adds the following unit tests OR does not need testing for this extremely good reason:

testing does not seem necessary
Commits

[x ] My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":

Subject is separated from body by a blank line
Subject is limited to 50 characters (not including Jira issue reference)
Subject does not end with a period
Subject uses the imperative mood ("add", not "adding")
Body wraps at 72 characters
Body explains "what" and "why", not "how"

Documentation

[ x] In case of new functionality, my PR adds documentation that describes how to use it.

When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.

Code Quality

[x ] Passes git diff upstream/master -u -- "*.py" | flake8 --diff

Codecov Report

Merging #3920 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master    #3920   +/-   ##
=======================================
  Coverage   77.53%   77.53%
=======================================
  Files         198      198
  Lines       15847    15847
=======================================
  Hits        12287    12287
  Misses       3560     3560

Impacted Files
Coverage Δ

airflow/utils/dag_processing.py
89.03% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5355909...f2371ed. Read the comment docs.

I'm +1 for this change


Electron 22/24/25 version bumps for CVE-2023-4863

Description of changes
Upstream released new versions on the 22.x / 24.x / 25.x branches for CVE-2023-4863 (among others, apparently).
Ref #254798
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

@ofborg build electron_22 electron_24 electron_25


Add yamllint and ansible-lint call in CI build

Add yamllint and ansible-lint call in CI build

closed in favor of #3


TorchRec Sharding Composability

Desired behavior
We want to make TorchRec sharding composable w/ other sharding/parallelism techniques. This practically means that after applying TorchRec sharding model characteristics remain the same (e.g. state_dict() doesn’t change) and we don’t effect non-sharded parts of a model, e.g.
m = Model(device="meta")
orig_keys = list(m.state_dict().keys())
m.ebc = torchrec.shard_embedding_bag_collection(m.ebc, ....)
sharded_keys = list(m.ebc.state_dict().keys())
assert orig_keys == sharded(keys) # at all ranks, even for table-wise sharded
Observation for nn.Module

state_dict() - recursive call via state_dict()
load_state_dict() : recursive call via local load() call, which calls _load_from_state_dict
named_parameters()/named_buffers() : calls named_modules
named_modules() : recursive call

As we don’t want to modify state_dict/named_parameters etc impls of non-sharded modules (including top-level module) we have to ensure that _modules is consistent
Motivations

Current implementation (of named_parameters()/named_buffers()/named_modules()) is consistent with nn.module defaults currently

This logic is tricky to get right

Removes need for custom code (e.g tricks in DMP and ShardedModules to keep module FQN consistent)
Currently state_dict does not contain state of all tensors

if a table is sharded table_wise, then only that rank’s state_dict sees this

easier integration with other distributed solutions (e.g FSDP for DHEN).

We need to share a similar semantics of what is returned by nn.Module APIs, and this will make things consistent.

We can also do this within DMP nn.Module overrides, but that route is much heavier and bug-prone

Proposed APIs
Use the trick employed by torch.fx.GraphModule of registering empty nn.Modules for handling state and have implementation of forward() call based on a different module hierarchy.
new APIs
def shard_embedding_modules(model, sharders, plan: EmbeddingShardingPlan):
   # This module will replace all child ebcs/fused_ebcs modules with sharded
   # variants based on the plans and init meta tensors.

   # it returns the unsharded modules and their replacements
   # this useful for identifying which modules are already sharded
   return sharded_parameter_names

def dlrm_parallelize(model,
   sharders=[EmbeddingBagCollectionSharder, FusedEmbeddingBagCollectionSharder, ...],
   embedding_plan: Optional[EmbeddingShardingPlan]):

   # This style of sharding will shard embedding modules with torchrec sharder
   # and everything else in a distributed data parallel fashion.

   if embedding_plan is None
      embedding_plan = EmbeddingPlanner(sharders, topology, ...)

   sharded_embedding_modules = shard_embedding_modules(model, plan)

   # DDP can be done in two ways
   # with DDP wrapper:
   # This will rely on DDP being composable

   model = DistributedDataParallel(model,
               params_and_buffers_to_ignore=[get_params_and_buffers(sharded_embedding_modules)]
           )
   modules = model.modules()

   # or replace underlying module tensors with DataParallelTensor
   for name, param in model.named_parameters():
       if param not in get_params_and_buffers(sharded_embedding_modules):
          model.register_parameter(name, DistributedParallelTensor(param))

# similar wrapper can be done with FSDP, or just do it in model definition

class ShardedEmbeddingBagCollection()
    def __init__(self):
        # modules to handle state
        self.embedding_bags: nn.ModuleDict = nn.ModuleDict()
        # these are sharded modules, Dict is not registered as sub-moduels
        self._lookups: Dict[str, nn.Module] ={}
        # Add parameters/buffers from self._lookups to self.embedding_bags

    def forward(self, KJT):
        # unchanged impl

Model Authoring
embedding_bag_configs = [
   EmbeddingBagConfig(
       name="t1", embedding_dim=4, num_embeddings=10, feature_names=["f1"]
   ),
   EmbeddingBagConfig(
       name="t2", embedding_dim=4, num_embeddings=10, feature_names=["f2"]
   ),
   EmbeddingBagConfig(
       name="t3", embedding_dim=4, num_embeddings=10, feature_names=["f3"]
   )
]
model = DLRMModel(ebc=EmbeddingBagCollection(embedding_bag_configs))
model = fuse_embedding_optimizer(model, ...) # recursively replaces model.ebc with FusedEBC

# using wrappers to parallelize
dlrm_parallelize(model)

# or explicitly do it
shard_embedding_modules(model,
   plan={"embedding_bag_configs":
    {"t1": ParameterSharding(sharding_type=ROW_WISE, placement=DEVICE, ranks=...)},
    {"t2": ParameterSharding(sharding_type=ROW_WISE, placement=UVM_CACHING, ranks=...)},
   })

print(model.named_parameters())
>>> [overarch..., linear,..., embedding_bag_configs.t1.weights: ShardedTensor]

opt = torch.optim.SGD(model.parameters(), lr=.02)

# if we do not use FusedEBC, then we expect that these ShardedTensors have
# a grad field (or maybe these are ShardedParameters), and optimizers will
# naturally work on top of them
# If we have FusedEBC, their grads will be None, and opt.step will be a no-op

print(model.state_dict()["embedding_bag_configs.t1.weight"])
>>> ShardedTensor(rank0: (tensor, size, offset), rank1: (tensor, size, offset))

# If t1 is placed on rank 1, rank 0's state dict will still see the ShardedTensor
# but it's local shards will be empty.

I have a question about the code snippet at the beginning of the previous comment.
m = Model(device="meta")
orig_keys = list(m.state_dict().keys())
m.ebc = torchrec.shard_embedding_bag_collection(m.ebc, ....)
sharded_keys = list(m.ebc.state_dict().keys())

assert orig_keys == sharded(keys) # at all ranks, even for table-wise sharded

What is m.ebc? Why are we supposed to compare m with m.ebc?
Do we mean the following instead?
m = Model(device="meta)
n = torchrec.shard_embeddings(m)
assert m.state_dict().keys() == n.state_dict().keys()

@wangkuiyi sorry for the delay :)
I think the snippet is a bit confusing, but the core API as landed is shard_embedding_modules https://github.com/pytorch/torchrec/blob/main/torchrec/distributed/shard_embedding_modules.py#L24
This will replace (module swap) the embedding modules and return a module that you can compare like you indicated
assert m.state_dict().keys() == n.state_dict().keys()
See https://github.com/pytorch/torchrec/blob/main/examples/golden_training/train_dlrm.py#L95 for an example in action
cc @YLGH

This is landed in master, and will be going out in the next stable release


visualizing publication data from wikiannotator

This issue is dependent on the data annotator for wikiannotator #6.
The idea would be to get the metadata for genes and then:

[ ] create a timeline-based visualization
[ ] create a top publication list

After fixing the app (transition from Streamlit to Dash)


Collaboration service (NOT FINISHED)

Milestone 5 Tasks:

[x] Connect matching to collab room
[x] Collaborative code editor
[x] Ensure that only matched users can enter collab (through start session button)
[x] Retrieve question based on matched preferences
[ ] Improve question display on Collab page to show Topic(s) and Difficulty
[ ] Improve topic drop down to only show topics already in database (and sort alphabetically)
[ ] Handle case where other user doesn't press Start Session button (show notification when user joins room)
[x] Implement disconnected mechanism - allow user to reconnect??
[x] Implement user quit - when one user quit, notify other user
[ ] Collaboration timer
[ ] Implement timeout - users have to extend session after 30mins, if user doesn't extend for a period of time (eg 30s then treat as user AFK and quit the session)

Milestone 6 Tasks (Enhanced code editor + Code execution):

[ ] Let user choose code editor theme (eg vs-light, vs-dark, etc)
[ ] Improve code editor to show Language
[ ] Implement code execution (Run button to execute code, Output tab to display result, Testcase tab to display testcases)

idk if is just me, but I got this error when I was in a collab session and I pressed f12 to open the console

idk if is just me, but I got this error when I was in a collab session and I pressed f12 to open the console

hmmm how did you get this error (like how can i try to recreate it)? oso the console just shows a warning which shouldn't be related to the error you are getting...

idk if is just me, but I got this error when I was in a collab session and I pressed f12 to open the console

@ChillinRage hmmm how did you get this error (like how can i try to recreate it)? oso the console just shows a warning which shouldn't be related to the error you are getting...

I just tried again. Seems like this error only happens for me when the browser is in full screen. I can send a screen recording if you'd like.

I might be losing my mind here, but just to check, are there any database for the collab service?


Release 2.1.1

[x] Create a release ticket in github. Copy this checklist.
[x] Make a release branch

VERSION="..."
VERSIONALPHA="$VERSION-alpha.0"
ISSUE="..."

git checkout -b release/$VERSION

[x] Update package.json version to $VERSIONALPHA and commit

git commit -m "release: Update package version to $VERSIONALPHA #$ISSUE"

[x] Tag with the alpha version

git tag $VERSIONALPHA

[x] Publish test npm version $VERSIONALPHA

npm publish

[x] Check package internals - contains all required files (lib/, README.md, ./djv.js, package.json, ...), no extra files
[x] Check with updated json-schema test suite - check generated report
[x] Check speed metrics
[x] Update package.json version to $VERSION and commit

git commit -m "release: Update package version to $VERSION #$ISSUE"

[x] Create a changelog

npm run changelog

[x] Update Readme
[x] Tag $VERSION

git tag $VERSION

[x] Merge release branch to master (via github)
[x] Publish version $VERSION

npm publish

[ ] Send update to gitter

https://gitter.im/korzio/djv?at=5b8e4b1ed8d36815e5d3a3e1


Python Command

Raw Paho MQTT implementation of command with minimalist infrastructure

It's ok to start, but I was expecting to see the code splitted into three layers:

Common. Helper funcions using in all samples, such as connecting with settings
Binders. Encapsulate topics and serialization formats
Clients. Just using the binders without any mqtt concepts

This kind of division is not very viable within the structure of the samples we're discussing here, which is why we have elected to implement samples directly with raw Paho instead. This is due to differences in how the MQTT client works in Python, as well as various language features that make mimicking the C# samples highly difficult and convoluted. I can explain the technical details in depth if you wish, please just reach out.
However, I can assure you that the pattern you desire will be reflected in the real SDK lite implementation, which we have already begun work on.


SyntaxError: Missing parentheses in call to 'print' resolved

SyntaxError: Missing parentheses in call to 'print' resolved .

Please merge

Fixes issue #3


Introducing the GF version 2020

This a new version that I am planning to be the base of the GF scheme in GEOS-5 onwards. The code was clean up; the three convective modes now have a unified treatment, contains several new physical and improved numerical aspects. It also has the "cold-pool" parameterization developed several months ago, including a 'convective memory' capability. Lastly, it also contains a lightning parameterization (Lopez, 2016 MWR), in-line with the deep convection module.
The convective tracer transport and wet removal modules were not touched.
Using the default configuration, it presents zero-diff with the "develop" tag.

@sdrabenh can we verify that these changes are zero-diff to the latest develop?

@sdrabenh can we verify that these changes are zero-diff to the latest develop?

@wmputman absolutely. I will test this PR tonight

This might be the largest-difference commit I've ever seen. Well-earned applause! 👏

@srfreita and @wmputman I ran my 1-day C48 tests on this and it looks like this PR is NOT 0-diff. Here is the breakdown:

develop amip vs PR amip: all 20 checkpoints differ
develop replay vs PR replay: all 20 checkpoints differ
PR amip vs PR 0-inc replay: all 20 checkpoints pass
My regression tests failed because I ran out of wallclock time - I will check later

Many differences are significant - not just round off errors. Please feel free to look at the files between the 2 experiments, "develop" and "pullreq" at:
/discover/nobackup/projects/gmao/g6dev/sdrabenh/ZeroDiffTests/PR_GF-2020/
For example, from the above directory issue the command:
cdo -diffn develop/scratch.1day_amip.dev/fvcore_internal_checkpoint pullreq/scratch.1day_amip.pul/fvcore_internal_checkpoint

Hi Scott,
Thanks for checking that. I only verified the 'geosgcm_prog' collection. I did it again using the "cdo" command, and those files are zero-diff:
/discover/nobackup/srfreita/geos5/x_zero_diff>cdo -diffn  new.holding/geosgcm_prog/201801/new.geosgcm_prog.20180101_0900z.nc4 tag.holding/geosgcm_prog/201801/tag.geosgcm_prog.20180101_0900z.nc4
cdo.exe diffn: Processed<PHONE_NUMBER> values from 82 variables over 2 timesteps ( 58.23s )
Saulo

Hi Scott,
Thanks for checking that. I only verified the 'geosgcm_prog' collection. I did it again using the "cdo" command, and those files are zero-diff:
/discover/nobackup/srfreita/geos5/x_zero_diff>cdo -diffn new.holding/geosgcm_prog/201801/new.geosgcm_prog.20180101_0900z.nc4 tag.holding/geosgcm_prog/201801/tag.geosgcm_prog.20180101_0900z.nc4
cdo.exe diffn: Processed<PHONE_NUMBER> values from 82 variables over 2 timesteps ( 58.23s )
Saulo
From: Scott Rabenhorst<EMAIL_ADDRESS>Reply-To: GEOS-ESM/GEOSgcm_GridComp<EMAIL_ADDRESS>Date: Friday, April 17, 2020 at 9:01 AM
To: GEOS-ESM/GEOSgcm_GridComp<EMAIL_ADDRESS>Cc: "Freitas, Saulo R. (GSFC-6100)[UNIVERSITIES SPACE RESEARCH ASSOCIATION]"<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: [EXTERNAL] Re: [GEOS-ESM/GEOSgcm_GridComp] Introducing the GF version 2020 (#264)
@srfreitahttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_srfreita&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=yMu-0sIcjVLx7j3wlgbVNkOhL1KlymKl8mox3qCukSo&s=f_kHGxlWMkSCoPzDLgX4M0wEua3fSD9D0WzsXTVsR3U&e= and @wmputmanhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_wmputman&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=yMu-0sIcjVLx7j3wlgbVNkOhL1KlymKl8mox3qCukSo&s=6MiV6JMdvJoylvD-dKURsLBx-dihekMU4F7EaYxNJi8&e= I ran my 1-day C48 tests on this and it looks like this PR is NOT 0-diff. Here is the breakdown:

  develop amip vs PR amip: all 20 checkpoints differ

  develop replay vs PR replay: all 20 checkpoints differ

  PR amip vs PR 0-inc replay: all 20 checkpoints pass

  My regression tests failed because I ran out of wallclock time - I will check later

Many differences are significant - not just round off errors. Please feel free to look at the files between the 2 experiments, "develop" and "pullreq" at:
/discover/nobackup/projects/gmao/g6dev/sdrabenh/ZeroDiffTests/PR_GF-2020/
For example, from the above directory issue the command:
cdo -diffn develop/scratch.1day_amip.dev/fvcore_internal_checkpoint pullreq/scratch.1day_amip.pul/fvcore_internal_checkpoint
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_GEOS-2DESM_GEOSgcm-5FGridComp_pull_264-23issuecomment-2D615230692&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=yMu-0sIcjVLx7j3wlgbVNkOhL1KlymKl8mox3qCukSo&s=lHAydprhrhJ1XIHcMsckypntWRi0OYtMtRe6Nboj2kg&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ANAREE7A54LVXGH73GFVOKDRNBHHJANCNFSM4MJ4UY3Q&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=yMu-0sIcjVLx7j3wlgbVNkOhL1KlymKl8mox3qCukSo&s=DvYXi6v6hQCRYa2rlowe3erJD983PneTPlFQRRN8ZLI&e=.

@srfreita that is the peculiar part to me. Yes, my 'geosgcm_prog' was 0-diff too, but many history collections are time averaged. I am looking that all checkpoint/restart files are identical.

Note if you like to have a bit of statistics, you can use nccmp as well. Looking at fvcore, the differences are:
Variable Group  Count      Sum      AbsSum         Min        Max     Range         Mean      StdDev
PE       /     988771 -25443.8 1.05188e+06    -68.8452    71.0404   139.886   -0.0257328     2.74037
PKZ      /     986729 -2.18554      94.871 -0.00577329 0.00559028 0.0113636 -2.21494e-06 0.000238932
PT       /     995327 -78.7735     7746.27    -6.32273    3.46943   9.79216 -7.91434e-05   0.0232624
U        /     995328  672.561     79769.8    -7.60102    7.71399    15.315  0.000675718    0.138249
V        /     995328 -1079.04     78643.4    -5.70421    6.17811   11.8823  -0.00108411    0.133763

Using a relative tolerance of 1% shows that PT, U, and V, are pretty different:
(1003) $ nccmp -dqfSB -T 1.0 develop/scratch.1day_amip.dev/fvcore_internal_checkpoint pullreq/scratch.1day_amip.pul/fvcore_internal_checkpoint
Variable Group  Count      Sum  AbsSum      Min     Max   Range        Mean   StdDev
PT       /         76 -14.5361 95.0545 -6.32273 3.46943 9.79216   -0.191264  1.63636
U        /     409797<PHONE_NUMBER>3.3 -7.60102 7.71399  15.315   0.0010412  0.19998
V        /     421934 -830.094 54014.4 -5.70421 6.17811 11.8823 -0.00196736 0.191165

@sdrabenh did the regression test finish?

@srfreita , until this passes zero-diff tests and regression we cannot move forward with the review

It also did not pass regression?
From: Bill Putman<EMAIL_ADDRESS>Reply-To: GEOS-ESM/GEOSgcm_GridComp<EMAIL_ADDRESS>Date: Monday, April 27, 2020 at 1:15 PM
To: GEOS-ESM/GEOSgcm_GridComp<EMAIL_ADDRESS>Cc: "Freitas, Saulo R. (GSFC-6100)[UNIVERSITIES SPACE RESEARCH ASSOCIATION]"<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: [EXTERNAL] Re: [GEOS-ESM/GEOSgcm_GridComp] Introducing the GF version 2020 (#264)
@srfreitahttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_srfreita&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=e0AaJ9CPJ8bTpvkk3dlX2Sg7Ly3X0amqN_OByOs4xuQ&s=tfd50ShRjsZwV_xYNrCNQ2rOE7N-SFBvKIC2jiZ9tqo&e= , until this passes zero-diff tests and regression we cannot move forward with the review
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_GEOS-2DESM_GEOSgcm-5FGridComp_pull_264-23issuecomment-2D620118448&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=e0AaJ9CPJ8bTpvkk3dlX2Sg7Ly3X0amqN_OByOs4xuQ&s=OOZJoQLt1JvHRrlaZX9RjD4sgWN_qUcisfiCAbJul8I&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ANAREEZT2N7SL2LYRR77YDLROW4SZANCNFSM4MJ4UY3Q&d=DwMCaQ&c=ApwzowJNAKKw3xye91w7BE1XMRKi2LN9kiMk5Csz9Zk&r=cPpuLDXNTBHW0MUOmIQYUg4XSRqkGIZXu19R8tK4g7c&m=e0AaJ9CPJ8bTpvkk3dlX2Sg7Ly3X0amqN_OByOs4xuQ&s=EvFgGTe3EOSrUxtLWVpyYDF4pI_O8la0PYh7Qw34g-k&e=.

@sdrabenh can you confirm this request this tag did not pass regression?

@srfreita and @wmputman I confirmed this did pass the regression testing. The problem was the restarts.

@wmputman Should we close this in favor of the edits you have in development?


Add Position direct control to the control board plugin

We want to add a low level, high frequency position control, corresponding to the Position Control Direct present in gazebo-yarp-plugins control board.

CC @traversaro

@traversaro while implementing the IPositionDirectinterface I'm not sure I've fully understood the documentation of the getRefPositionRaw method. It mentions that:

This is the dual of setPositionsRaw and shall return only values sent using
IPositionDirect interface.
If other interfaces like IPositionControl are implemented by the device, this call
must ignore their values, i.e. this call must never return a reference sent using
IPositionControl::PositionMove.

I've seen that controlboard plugin in gazebo-yarp-plugins implements this methods and does not check the control mode.
Should I implement this method even if I will implement  the IPoisitionControl interface afterwards? Maybe just checking the actual control mode before returning the position reference value could be enough?

By looking at the gazebo-yarp-plugin implementation, I've seen that the force computation given the position reference is made with a PID: https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/src/ControlBoardDriver.cpp#L744-L749
Has this issue a dependency on the implementation of IPidControl?

By looking at the gazebo-yarp-plugin implementation, I've seen that the force computation given the position reference is made with a PID: https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/src/ControlBoardDriver.cpp#L744-L749
Has this issue a dependency on the implementation of IPidControl?

No, IPIDControl only permits to set the gains, but if we do not want to change them at runtime, we can just set them only from configuration files.

@traversaro while implementing the IPositionDirectinterface I'm not sure I've fully understood the documentation of the getRefPositionRaw method. It mentions that:

This is the dual of setPositionsRaw and shall return only values sent using
IPositionDirect interface.
If other interfaces like IPositionControl are implemented by the device, this call
must ignore their values, i.e. this call must never return a reference sent using
IPositionControl::PositionMove.

I've seen that controlboard plugin in gazebo-yarp-plugins implements this methods and does not check the control mode.
Should I implement this method even if I will implement the IPoisitionControl interface afterwards? Maybe just checking the actual control mode before returning the position reference value could be enough?

The documentation is clear, probably the gazebo-yarp-plugins behaviour predates that version of the documentation. Let's implement what the documentation prescribe.

By looking at the gazebo-yarp-plugin implementation, I've seen that the force computation given the position reference is made with a PID: https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/src/ControlBoardDriver.cpp#L744-L749
Has this issue a dependency on the implementation of IPidControl?

No, IPIDControl only permits to set the gains, but if we do not want to change them at runtime, we can just set them only from configuration files.

Ok thank you, I misunderstood the scope of such interface. I will remove the dependency on issue #91 and put it back on backlog.

PID class is implemented in gz-math: https://github.com/gazebosim/gz-math/blob/gz-math7/src/PID.cc

@traversaro do you suggest porting this logic? Should we support only the POSITION_CONTROL way?

Yes, that would be great. I think we should support just POSITION_CONTROL, that I guess it should be aligned with how the gains are specified for the real robots (but I am not 100% sure about this).

@traversaro I've a doubt when reading how the PID parameters are initialized in gazebo-yarp-plugins.
In https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/src/ControlBoardDriver.cpp#L1176-L1179 the gazebo PID parameters are set by calling, in case of revolute joints, the convertUserGainToGazeboGain method.
This method does the opposite of what I was thinking: it converts the scale parameter from radians to degrees: https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/src/ControlBoardDriver.cpp#L1772-L1773
Is it correct? I was thinking that user uses degrees and gazebo radians.

I think what those methods are doing is correct, but the use of those functions are misleading. The unit of measurements used in YARP config files for revolute joints is $\frac{Nm}{deg}$, while the one used in the code is  $\frac{Nm}{[rad]}$. So, converting a gain from $\frac{Nm}{deg}$ to $\frac{Nm}{[rad]}$ uses the same formula used to convert a quantity from $[rad]$ to $deg$ (try to write it down), but indeed it is quite misleading to use the same function, probably we could use a function called convertDegreeGainsToRadianGains, or something similar to avoid confusion.

I think what those methods are doing is correct, but the use of those functions are misleading. The unit of measurements used in YARP config files for revolute joints is Nmdeg, while the one used in the code is Nm[rad]. So, converting a gain from Nmdeg to Nm[rad] uses the same formula used to convert a quantity from [rad] to deg (try to write it down), but indeed it is quite misleading to use the same function, probably we could use a function called convertDegreeGainsToRadianGains, or something similar to avoid confusion.

😱
Yep definitely a better name!

@traversaro what can be a useful test to validate the direct position implementation? Applying a position reference to the single pendulum (via unit test or using yarpmotorgui) and verifying by reading the actual joint position that it follows the setpoint?

@traversaro what can be a useful test to validate the direct position implementation? Applying a position reference to the single pendulum (via unit test or using yarpmotorgui) and verifying by reading the actual joint position that it follows the setpoint?

Yes, we can check that the average and/or max tracking error is below a given threshold.

@traversaro I'm getting closer to have a first working implementation, however I'm encountering strange behaviors using the yarpmotorgui.
Here is when I simulate the single pendulum using gazebo classic and gazebo-yarp-plugins:
https://github.com/robotology/gz-sim-yarp-plugins/assets/57228872/72c2b183-1df0-426e-a8fe-ce50d37232e5
Here what happens with Gazebo and gz-yarp-plugins:
https://github.com/robotology/gz-sim-yarp-plugins/assets/57228872/0b19e1d0-d45b-42ef-8677-9ffebf53e051
As you can see the value the yarpmotorgui reports when trying to change the reference position is really large (also the slider it is not initialized in the middle position, but on the left end).

The issue appears to be connected to missing joint limits that make yarpmotorgui position slider resize badly.
By looking at the code I have not understood if the joint limits (position and velocity) are connected with the ones specified in the urdf (e.g. https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/tutorial/model/single_pendulum/model.urdf#L38) or if they are set from yarp ports calling SetLimits() method from IControlLimits:
https://github.com/robotology/gazebo-yarp-plugins/blob/cb3521f7543a5a07e2a65a82dedd2eb2ec2cea12/plugins/controlboard/include/yarp/dev/ControlBoardDriver.h#L318-L319

@traversaro who should set these limits and based on what values? I could suppose that the limit should be initialized from the values of the gazebo model (that in turn are specified in the model sdf/urdf). Then, using the IControlLimits interface it will be possible to change them (this should in turn call also SetPositionLimits )

Answer: there a distinction between (simulated) hardware limits and software limits:

Hardware limits are the ones specified in robot description (<limit> tag in model sdf/urdf) and are managed by the Gazebo simulator
Software limits are limits, generally more conservative that are managed by the controlboard and they are configured in the control board ini configuration file (e.g. for ergocub model https://github.com/icub-tech-iit/ergocub-software/blob/10670acf8120bdaad19c6cb01388eff555f4f356/urdf/ergoCub/conf/gazebo_ergocub_left_leg.ini#L50C1-L53C46) specified with the [LIMITS] group.

I'm not able to read the gazebo joint limits, since from their joint API wrapper it is marked as TODO (it was possible in gazebo classic) and from the ECM it seems not querable since the joint limit component is missing (only the components to set new limits are available).
Getting these limits would be useful since we should check if the sw limits are coherent with the hw limits set in the model description and in case the LIMITS group is not defined from the control board we could use them as a fallback.
Related robotics stack exchange question: https://robotics.stackexchange.com/questions/108512/how-to-get-joint-position-limits-in-gazebo

Ok, but I guess for now we can just use the [LIMITS] one from the .ini file, right?

Ok, but I guess for now we can just use the [LIMITS] one from the .ini file, right?

yep!

In commit e9a8a63602a494385dfd18e379b1259bdd6264bc the joint limits are configured from the configuration file.
In the following recording a demo of the control mode:
https://github.com/robotology/gz-sim-yarp-plugins/assets/57228872/470cbccb-579c-4758-9ef7-40c3cb10fbbf
It still remains the fact that in yarpmotorgui the encoder block is red (what does it mean?) and there is a steady state error decreasing with decreasing effect of the joint torque.
The PID tuning is the following:
https://github.com/robotology/gz-sim-yarp-plugins/blob/e9a8a63602a494385dfd18e379b1259bdd6264bc/tutorial/single_pendulum/conf/gazebo_controlboard.ini#L7-L18

the encoder block is red (what does it mean?)

Not sure, I would need to check the code.

It is because the motion appears not "done". Basically it depends on the output of checkMotionDone. See https://github.com/robotology/yarp/blob/6ca59397df63374d7112b977b9bd425208b6f5c6/src/yarpmotorgui/jointitem.cpp#L1560-L1565 and https://github.com/robotology/yarp/blob/6ca59397df63374d7112b977b9bd425208b6f5c6/src/yarpmotorgui/partitem.cpp#L2234-L2292

Thank you!!

As suggested in https://github.com/robotology/gz-sim-yarp-plugins/issues/82#issuecomment-1959469967, in a11d5f4ff4b0a694756fbc292e9461b9e95b8e54 I've implemented a unit test in which a sinusoidal position reference is set for the single pendulum joint, and the tracking error (instantaneous in this case) is checked that is below a threshold.
Here is a video of the simulation performed in the test:
https://github.com/robotology/gz-sim-yarp-plugins/assets/57228872/e7df1f44-c977-42bf-bab6-7d1f80116550

I would increase the amplitude of the sine, at least for scenographic effect. :D

I would increase the amplitude of the sine, at least for scenographic effect. :D

ahah sure! I will also change the tracking error expectation to check the average error, since I'm encountering issues in a very few steps having a tracking error much higher than the average (like 5 steps in 5000 fail the expectations).

I would increase the amplitude of the sine, at least for scenographic effect. :D

ahah sure! I will also change the tracking error expectation to check the average error, since I'm encountering issues in a very few steps having a tracking error much higher than the average (like 5 steps in 5000 fail the expectations).

Interesting! Probably we can think of investigating this in the future, for example plotting this. However, for the time being we can proceed.


Executing using INT8 operations

On NVIDIA sm80 and later GPUs, INT8 accelerated compute is available. Is there a way to use 8 bit quantized weights with 8 bit compute?

@stduhpf / @Green-Sky / idk, I'm interested as well, is there an update on this? I've got a bunch of 90hx's waiting to be unleashed with int8 (no fp).

This should be handled by the GGML library, not by stable-diffusion.cpp. And if I'm not mistaken, the GGML-Cuda backend should already take adventage of int8 accelerated compute? (according to this https://github.com/ggerganov/llama.cpp/issues/9578)

@JakoDel interesting hardware, I wonder how well it does with ml compute. As @stduhpf said, cuda is supported, but maybe not your hardware specifically. In any case, you can go ahead and try it.

@JakoDel interesting hardware, I wonder how well it does with ml compute. As @stduhpf said, cuda is supported, but maybe not your hardware specifically. In any case, you can go ahead and try it.

I've ordered 3 from china, I'll let you guys know how it goes. it stings that the 170hx's with 1.5tb/s were thrice as expensive lol.


[bug] error: failed to get serde as a dependency of package app v0.1.0 (/Users/project)

Describe the bug
When i Setting Up macOS and run npm run tauri:serve throw error error: failed to get serde as a dependency of package app v0.1.0 (/Users/project)
Reproduction
No response
Expected behavior
No response
Platform and versions
Operating System - Mac OS, version 12.0.1 X64

Node.js environment
  Node.js - 14.15.4
  @tauri-apps/cli - 1.0.0-rc.4 (outdated, latest: 1.0.0-rc.7)
  @tauri-apps/api - Not installed

Global packages
  npm - 6.14.10
  pnpm - Not installed
  yarn - 1.22.10

Rust environment
  rustup - 1.24.3
  rustc - 1.59.0
  cargo - 1.59.0
  toolchain - stable-x86_64-apple-darwin

App directory structure
/dist
/node_modules
/public
/src-tauri
/dist_electron
/.git
/src

App
  tauri - 1.0.0-rc.2 (no lockfile)
  tauri-build - no manifest (no lockfile)
  tao - no manifest (no lockfile)
  wry - no manifest (no lockfile)
  build-type - bundle
  CSP - unset
  distDir - Set automatically by Vue CLI plugin
  devPath - Set automatically by Vue CLI plugin
  framework - Vue.js (Vue CLI)
  bundler - Webpack

Cargo.toml:
[package]
name = "app"
version = "0.1.0"
description = "A Tauri App"
authors = ["you"]
license = ""
repository = ""
default-run = "app"
edition = "2021"
rust-version = "1.57"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[build-dependencies]
tauri-build = { version = "1.0.0-rc.1", features = [] }

[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
tauri = { version = "1.0.0-rc.2", features = ["api-all"] }

[features]
# by default Tauri runs in production mode
# when `tauri dev` runs it is executed with `cargo run --no-default-features` if `devPath` is an URL
default = [ "custom-protocol" ]
# this feature is used used for production builds where `devPath` points to the filesystem
# DO NOT remove this
custom-protocol = [ "tauri/custom-protocol" ]

Stack trace
No response
Additional context
No response

Please fill out "Platform and versions" like asked with the output of tauri info
Can you post your Cargo.toml? Do you have serde inside of that?
Can you see if running cargo update in your src-tauri dir prints any (more extensive) errors? If so, please post them too.

Please fill out "Platform and versions" like asked with the output of tauri info
Can you post your Cargo.toml? Do you have serde inside of that?
Can you see if running cargo update in your src-tauri dir prints any (more extensive) errors? If so, please post them too.

Updated

Please paste the full error message (also try removing src-tauri/Cargo.lock or running cargo update). It's easier to help you in Discord if you can join our server :) this error should never happen to a new project.
Also, your project path looks weird (/Users/project). Could be a permission issue.


fix(timout): Increasing API timeout from 30 seconds to 5 minutes

:tada: This PR is included in version 1.31.8 :tada:
Your semantic-release bot :package::rocket:


Add a CONTRIBUTING Guide

We should add a Contributing Guide with the steps required to locally setup and needed to be followed by a contributor. This will help contributors know the requirements and contribute better.

Closing this issue as the PR has been merged.


kube-otel-stack daemonset not propagating to ALL nodes

Situation: when trying to deploy daemonset on kube-otel-stack for logscollector, I expect that the collector to appear on EVERY node.  (note, I have a stateful set metrics collector and a daemonset logs collector)
Problem: it doesn't actually appear on every node. whereas on the opentelemetry collector helm chart, it DOES appear on every node.

Upon inspection, I noticed that an extra toleration of NoSchedule op=Exists is in the otel-collector helm chart which allows for a collector on EVERY node.

how would we fix this issue?

is it as simple as adding an extra toleration to propagate to the collector? would this require re-formatting how this helm chart is written?

/assign


Added roadsandlibraries

Pull Request Template
Description
Gave solution for Roads and Libraries challenge from hackerrank.
Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.
Checklist:

[ yes] My code follows the style guidelines of this project
[yes ] I have performed a self-review of my own code
[ yes] I have commented my code, particularly in hard-to-understand areas
[ yes] My changes generate no new warnings

Thanks @jnaneswar for contributing.
Can you also update the problem and solution links into the README file ?

can u add this repo to hacktober fest

Can you update Readme file ?
Else I have to reject your PR

Let me edit the readme file


CSS not loaded

Issue description

components: Button
reactstrap version 5.0.0-alpha.4
react version 16.2.0
bootstrap version 4.0.0-beta.3

What is happening?
I am importing the button, I am rendering it, I can see the button but the style is not applied. The css is imported I can see it, but is not applied to the button

I am using it inside electron.
My dependencies:
    "animated": "^0.2.0",
    "auto-launch": "^5.0.1",
    "bootstrap": "^4.0.0-beta.3",
    "classnames": "^2.2.5",
    "devtron": "^1.4.0",
    "electron-debug": "^1.2.0",
    "electron-store": "^1.2.0",
    "electron-sudo": "^4.0.12",
    "history": "^4.6.3",
    "lodash": "^4.17.4",
    "mdi": "^2.0.46",
    "prop-types": "^15.5.10",
    "react": "^16.2.0",
    "react-dom": "^16.2.0",
    "react-google-maps": "^9.4.5",
    "react-hot-loader": "3.1.3",
    "react-jss": "^7.0.2",
    "react-redux": "^5.0.6",
    "react-router": "^4.2.0",
    "react-router-dom": "^4.2.2",
    "react-router-redux": "^5.0.0-alpha.6",
    "react-transition-group": "^2.2.0",
    "reactstrap": "^5.0.0-alpha.4",
    "recompose": "^0.26.0",
    "redux": "^3.7.2",
    "redux-form": "^7.2.0",
    "redux-saga": "^0.16.0",
    "redux-thunk": "^2.2.0",
    "semantic-ui-flag": "^2.2.11",
    "source-map-support": "^0.5.0",
    "sudo-prompt": "^7.1.1",
    "warning": "^3.0.0",
    "winston": "^2.3.1"

Code:
// @flow
import React from 'react';
import { ConnectedRouter } from 'react-router-redux';
import { Container, Button } from 'reactstrap';

type RootType = {
  store: Object,
  history: Object
};

export default function Root({ store, history }: RootType) {
  return (
    <Container>
      <Button color="primary">primary</Button>

    </Container>
  );
}

Imported css in index.js as:
import 'bootstrap/dist/css/bootstrap.css';

Thanks, that was helpful. Why
import '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css';
is work, but in the official documentation is import 'bootstrap/dist/css/bootstrap.css';  which is not working?

pretty helpful thanks @jalanga

./src/index.js
Line 3:  Unexpected '!' in '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css'. Do not use import syntax to configure webpack loaders  import/no-webpack-loader-syntax
Search for the keywords to learn more about each error.
i am getting this error while importing .how to fix this .

@youngheart12 See https://github.com/benmosher/eslint-plugin-import/blob/master/docs/rules/no-webpack-loader-syntax.md

Are you using react js?
Sent from Yahoo Mail on Android
On Fri, Feb 22, 2019 at 2:10 PM, Dilip<EMAIL_ADDRESS>wrote:
./src/index.js
Line 3: Unexpected '!' in '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css'. Do not use import syntax to configure webpack loaders import/no-webpack-loader-syntax
Search for the keywords to learn more about each error.
i am getting this error while importing .how to fix this .
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

I fixed my css issues with a code..in the package json file..  Am not with my pc now.  I will forward it later if you need.
Sent from Yahoo Mail on Android
On Sat, Feb 23, 2019 at 2:31 AM, Chinonson Enam<EMAIL_ADDRESS>wrote:   Are you using react js?
Sent from Yahoo Mail on Android
On Fri, Feb 22, 2019 at 2:10 PM, Dilip<EMAIL_ADDRESS>wrote:
./src/index.js
Line 3: Unexpected '!' in '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css'. Do not use import syntax to configure webpack loaders import/no-webpack-loader-syntax
Search for the keywords to learn more about each error.
i am getting this error while importing .how to fix this .
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

Having same problem as @youngheart12. Here's my webpack config file:
const getStyleLoaders = (cssOptions, preProcessor) => {     const loaders = [       isEnvDevelopment && require.resolve('style-loader'),       isEnvProduction && {         loader: MiniCssExtractPlugin.loader,         options: Object.assign(           {},           shouldUseRelativeAssetPaths ? { publicPath: '../../' } : undefined         ),       },       {         loader: require.resolve('css-loader'),         options: cssOptions,       },       {         // Options for PostCSS as we reference these options twice         // Adds vendor prefixing based on your specified browser support in         // package.json         loader: require.resolve('postcss-loader'),         options: {           // Necessary for external CSS imports to work           // https://github.com/facebook/create-react-app/issues/2677           ident: 'postcss',           plugins: () => [             require('postcss-flexbugs-fixes'),             require('postcss-preset-env')({               autoprefixer: {                 flexbox: 'no-2009',               },               stage: 3,             }),           ],           sourceMap: isEnvProduction ? shouldUseSourceMap : isEnvDevelopment,         },       },     ].filter(Boolean);     if (preProcessor) {       loaders.push({         loader: require.resolve(preProcessor),         options: {           sourceMap: isEnvProduction ? shouldUseSourceMap : isEnvDevelopment,         },       });     }     return loaders;   };

Having same issues as @youngheart12 , am a beginner in react and the accepted answered above is using an old config file from webpack

There is nothing special or reactstrap specific for importing CSS (such as bootstrap's CSS).
The easiest way is to just link to the CSS hosted on a CDN as recommended by bootstrap: https://getbootstrap.com/docs/4.3/getting-started/introduction/#css
If you want to require/import CSS, then you need to have your build tool handle .css (or even .scss/.sass) files, else it will not know what to do with the file and probably assume that it is javascript (and blow up since CSS is not JavaScript).
In webpack, a loader (or a few loaders) are needed. You can find out more about webpack loaders and loading CSS via webpack on webpack's documentation site: Loaders and css-loader
Using something like CRA will preconfigure your webpack in a common/opinionated way. It supports loading CSS out of the box.
The issue that @youngheart12 and @drindc24 are running into is that they have a project which specific forbids using inline loaders. An inline loader is a modification to the import statement to make the import be a non-standard import. It adds which loader you want to use when loading the file to the file name string.
For instance this is a legit import which many systems understand:
import 'bootstrap/dist/css/bootstrap.css';

this is a webpack specific import which webpack understands
import '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css';

The problem with the webpack specific import is that you are coupling your imports to a specific build tool (which is considered bad practice).
Because of this, an eslint rule was created to find webpack specific import instances and throw an error. This eslint rule is in @youngheart12 and @drindc24 project, they can disable it to allow them to do the inline imports OR they can add the imports to their webpack configuration (see css-loader).

Thank you Evan sharp. Now I got to know where I was making a
mistake.Thankyou once again Evan.
On Mon, 25 Feb 2019 at 10:59 PM, Evan Sharp<EMAIL_ADDRESS>wrote:

There is nothing special or reactstrap specific for importing CSS (such as
bootstrap's CSS).
The easiest way is to just link to the CSS hosted on a CDN as recommended
by bootstrap:
https://getbootstrap.com/docs/4.3/getting-started/introduction/#css
If you want to require/import CSS, then you need to have your build tool
handle .css (or even .scss/.sass) files, else it will not know what to do
with the file and probably assume that it is javascript (and blow up since
CSS is not JavaScript).
In webpack, a loader (or a few loaders) are needed. You can find out more
about webpack loaders and loading CSS via webpack on webpack's
documentation site: Loaders https://webpack.js.org/concepts/loaders/
and css-loader https://webpack.js.org/loaders/css-loader/
Using something like CRA will preconfigure your webpack in a
common/opinionated way. It supports loading CSS out of the box.
The issue that @youngheart12 https://github.com/youngheart12 and
@drindc24 https://github.com/drindc24 are running into is that they
have a project which specific forbids using inline loaders
https://webpack.js.org/concepts/loaders/#inline. An inline loader is a
modification to the import statement to make the import be a non-standard
import. It adds which loader you want to use when loading the file to the
file name string.
For instance this is a legit import which many systems understand:
import 'bootstrap/dist/css/bootstrap.css';
this is a webpack specific import which webpack understands
import '!style-loader!css-loader!bootstrap/dist/css/bootstrap.css';
The problem with the webpack specific import is that you are coupling your
imports to a specific build tool (which is considered bad practice).
Because of this, an eslint rule was created to find webpack specific
import instances and throw an error. This eslint rule is in @youngheart12
https://github.com/youngheart12 and @drindc24
https://github.com/drindc24 project, they can disable it to allow them
to do the inline imports OR they can add the imports to their webpack
configuration (see css-loader https://webpack.js.org/loaders/css-loader/
).
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/reactstrap/reactstrap/issues/778#issuecomment-467101605,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AgCX5E024G9QVl-ixWzqGEnzysSPTFhVks5vRB1pgaJpZM4Rem_m
.

from where i can get the options of disabling eslint rules

import 'bootstrap/dist/css/bootstrap.css'; did for me.
Example:
import React from 'react';
import {
    Dropdown,
    DropdownToggle,
    DropdownMenu,
    DropdownItem
} from "reactstrap";
import 'bootstrap/dist/css/bootstrap.css';

export default class MyCustomDropdown extends React.Component {

    toggle = () => {
        this.setState(prevState => ({
            dropdownOpen: !prevState.dropdownOpen
        }));
    }

    render() {
        return <Dropdown isOpen={this.state.dropdownOpen} toggle={this.toggle}>
            <DropdownToggle caret>Dropdown</DropdownToggle>
            <DropdownMenu>
                <DropdownItem>Foo Action</DropdownItem>
                <DropdownItem>Bar Action</DropdownItem>
                <DropdownItem>Quo Action</DropdownItem>
            </DropdownMenu>
        </Dropdown>;
    }
}

@mkalish For some reason when I ran a build the CSS was loss. I decided to import the bootrap css to my top level app component. Thanks this helped!


win7 python 3.6  ( Building wheels for collected packages: ujson )

win 7
python 3.6
i try install with:
pip install -U steem

Building wheels for collected packages: ujson
Running setup.py bdist_wheel for ujson ... error
Failed building wheel for ujson
Running setup.py clean for ujson
Failed to build ujson
Installing collected packages: ujson
Running setup.py install for ujson ... error
Exception:
Traceback (most recent call last):
File "C:\Users\hayrettin\Anaconda3\lib\site-packages\pip\compat_init_.py",
line 73, in console_to_str
return s.decode(sys.stdout.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfd in position 48: invalid
start byte

https://steemit.com/utopian-io/@holger80/how-to-install-steem-python-for-windows
conda install ujson


Correct Existing Natural Keys 2 of 3 [Intermediate] - Conditional Uniqueness

Correct or assert existing natural keys on objects where conditional uniquness is a requirement or concern.
### Tasks
- [ ] **Service:** Unique together: `(device, name)` and `(vm, name)` with natural key `(name, device, vm)`; for duplication use other fields to help add to name uniqueness - 2
- [ ] **Device Type:** A stretch goal is to rename `model` to `name`
- [ ] **Device:** Configurable natural key/unique, default to PK
- [x] **Computed Field:** Make `key` unique
- [ ] **VirtualChassis:** Make `name` unique, reject migration if duplicate (don't want to rename VCs) - 2
- [ ] **ScheduledJob:** Make `name` unique, enumerate if duplicate, update help text and auto-assign name (the uuid) for one-off jobs
- [ ] **JobResult:** Ensure `pk` is called out as the natural key
- [ ] **Job:** (see below)
- [ ] **ExportTemplate:** Move to `unique_together` on just `content_type` and `name`, reject migration if duplicate
- [ ] **ConfigContextSchema** Move to `name` unique, reject migration if duplicate
- [ ] **ConfigContext** Move to `name` unique, reject migration if duplicate

Job

Relax unique on name because it will be make unique together with grouping: unique_together = ["grouping", "name"]
Make the natural key class_path which is unique
Other fields (git repository, source, etc) being removed in #765

VLAN Group: discussed in #2909
Circuit: Good enough for now with ["provider", "cid"]
Circuit Termination: Good enough with ["circuit", "term_side"]
Cable: Good enough for now, just could use some UX improvements for describing it
GitRepository name good enough for now

Remaining stories carved into #3845, #3846, #3847, #3848, #3849


Filter our suspended threads from CPU hotpath profiler

Context
The main goal CPU hotpaths profiler is to list the CPU hotpaths, in an inclusive percentage of CPU usage. Currently, the way this profiler works is by iterating over callstacks for all managed threads at a fixed time interval. While this approach gives quite accurate results of what threads are doing for a very small overhead, it lists callstacks for working threads as well as for suspended threads. Having suspended threads callstacks mixed in the results dilutes where the actual CPU power is being spent on.
This is not trivial. In fact, most CPU profilers don't offer this feature.
Solutions
Find a way to filter out suspended thread from the analysis.
Use RuntimeThreadSuspended/Resumed callbacks
This approach as several drawbacks:

It requires one more profiling flag COR_PRF_MONITOR_THREADS (and possibly a costly one)
It complexity the code quite a bit
Since we already suspend the runtime to take the stacks snapshots, we need to disambiguate threads suspended because of this pause

Call the OS API to know if a thread is suspended

No straightforward on linux and macos.
Possibly quite resource intensive
It would complexity the code quite a bit and make the profiler OS-dependent.

Use context information to (approximately) tell if a thread is suspended
Inspired from this: https://www.usenix.org/legacy/publications/library/proceedings/coots99/full_papers/liang/liang_html/node10.html
Todo

[x] Factorize the way we take stack snapshot to ease implementations (with trait?)
[x] Add profiler parameter (default is filter out suspended threads)
[x] BONUS: Refactor to use Tree structure

Before:

After, but with filtering disabled:

After, with filtering enabled and using instruction pointer to detect suspended threads:

After, with filtering enabled and using register context hash to detect suspended threads:

I'm not sure which is the most accurate way of filtering out suspended threads

I also took the opportunity to make use of the Tree structure. It has several benefits:

Simplifies the CPU hotpaths profiler code
Fixes the HTML report layout issue(s?)
Build tree using Function IDs, and only compute method names at the end, and far less times (once per TreeNode)

I also write the HTML report from the thread, as it simplifies a lot the borrowing boilerplate stuff required by the Rust borrow checker when dealing with thread shared resources.


[ACM-5527] Update managed cluster label allowlist

Jira issue: https://issues.redhat.com/browse/ACM-5527

Removing cluster.open-cluster-management.io/clusterset and name label from ignore label list.
Setting default label within the ACM overview dashboard to use the name label.

/hold

/test test-e2e

/unhold


[Issue-2352] Do not show token when standing on All accounts mode in case token does not get balance

Related issue: #2352

🚀 Deployed on https://pr-3497--sw-web-runner.netlify.app


Exported file contain Javascript content

Prerequisites

[ ] Able to reproduce the behaviour outside of your code, the problem is isolated to Laravel Excel.
[X] Checked that your issue isn't already filed.
[X] Checked if no PR was submitted that fixes this problem.

Versions

PHP version: 7.1.7
Laravel version: 5.7.26
Nova version: 1.3.1
Package version: 3.1 & 1.1

Description
When i try to export file, content is Javascript content
Steps to Reproduce
Expected behavior:
My data exported
Actual behavior:
Javascript code inside content
Additional Information
On my project i have json editor plugin loaded (cf: https://github.com/json-editor/json-editor). it seems it's Jsoneditor in content file. Maybe it's a conflict ?

If you disable json-editor, is the problem gone?
It seems like json-editor is always appending their javascript code, even when it's no needed (download response)

If i disable json-editor, i have the same problem than here : https://github.com/Maatwebsite/Laravel-Nova-Excel/issues/11 (but can't solve it)

Hi, I'm a colleague of Ange7 and I've figured out what was the problem, if the php.ini directive arg_separator.output is not ? (in our case it was & for some reason) the url created in DownloadExcel would be invalid.
And then I don't know why but the $this->validate of the ExcelController would redirect to some other nova-component, hence the javascript content inside the file.
We fixed it with an ini_set('arg_separator.output', '&'); but maybe it should be added to the controller directly.


Bug Of "drawable://" + R.drawable.img

if may cause some bug while I am loading two different resource but returns same bitmap, android 4.4

because you add new drawable but it has cache use last id

I rebuild it, but problem resolved after I reinstall my app

Your disk has cached the bitmap use last id，but the id is‘t the right because of your adding.you may need to clear you disk cache and don't use cache on "drawable://"


Cluster scalling API

Hello!
Has not client supported cluster scaling API yet?
https://qdrant.tech/documentation/distributed_deployment/#cluster-scaling

Follow the status: https://github.com/qdrant/rust-client/pull/33

part of https://github.com/qdrant/rust-client/pull/45

fixed in 1.2.0


generated column 'variant_listing_config' in table 'product'

hi,
first off thank you a lot for providing this script.
I want to use it to sync a production instance with a development instance under dockware.
SW6 is <IP_ADDRESS>
Source DB is mysqld Ver 10.3.37-MariaDB-0ubuntu<IP_ADDRESS>, mysqldump  Ver 10.19 Distrib 10.3.37-MariaDB
Target DB is mysqld  Ver 5.7.40-0ubuntu<IP_ADDRESS>. (dockware)
Dump was created with --gdpr option.
On import i get error:
The value specified for generated column 'variant_listing_config' in table 'product' is not allowed
On removing the product table inserts from the dump, the import finished without errors.
Yet users table was empty (had to create new admin user on console)
I suspect MariadDB is not supported by this script?
MariaDB 10.3.22 or higher is officially supported by shopware, so it should be 'Standard DB' ?
Could we extend script to write product table into temporary table without column  variant_listing_config and dump that table instead? Or maybe a view?
Or maybe mysqldump version is to blame?
Please advise howto proceed.
TIA
tom

This is related to an incompatibility between MariaDB and MySQL for generated columns, see the MySQL issue for that. I recommend not switching between the two, if possible.
You could also try generating the dump with mysqldump from MySQL instead of MariaDB's mysqldump.
Unfortunately, hard to fix on our side.

Thank you for your prompt reply.

incompatibility between MariaDB and MySQL for generated columns

That would have been great to have known before. Switching to another DB Engine on production without being able to do a proper dump will be tricky.

try generating the dump with mysqldump from MySQL instead of MariaDB's mysqldump

will try that. Thanks.


Cross-db compatibility guidelines?

I would find it helpful to have some guidelines for making code portable to both level.js and leveldown. Basically, I tried to port some leveldown code into level-browserify, thinking that it would "just work," but it turned out that it didn't. Is that something that we could add into the README of this project?
As far as specifics, the only thing that I'm aware of is that level.js likes ArrayBuffers whereas leveldown likes Buffers. Are there other details worth mentioning?

Hmm, tbh I'm not aware of any other differences, the goal should be really to make them as similar as possible.

Though this only starts to show with big datasets (in the thousands), IMO the biggest and most problematic difference is the lack of backpressure. You have to consume data as fast as it comes out, or accept memory spikes. IndexedDB cursors (and therefor level iterators and read streams) can't be paused, or in other words, they push, rather than being pulled from.

As far as specifics, the only thing that I'm aware of is that level.js likes ArrayBuffers whereas leveldown likes Buffers.

The next release of level-browserify, with level-js@3, will also prefer Buffers. In addition, it will support binary keys, have snapshot guarantees and pass the full abstract-leveldown test suite. Please see the (work-in-progress) upgrade guide in https://github.com/Level/level-js/pull/143 for more details.


Cisco ASA Legacy connector still receiving logs after migration to AMA connector

Hello everyone, so recently I have migrated my machines from an environment with Cisco ASA from the Legacy Connector to the new AMA connector.
I uninstalled the legacy connector completely from my machine, and the Workspace settings for the Sentinel environment also show there are no more legacy agents connected or collections rules active for these.
However, when I go to the data connector section, both the legacy and AMA connector still show up as connected with both receiving the same amount of data.
I would like to disconnect and remove the legacy connector, however I can't figure out how to do this, can anyone help with this?
Note: I am receiving all logs properly from the new AMA connector and there is no other data being collected as far as I can see, it just seems like the legacy and the AMA connector reference the same data table causing this issue.

Hi @thom2804, Thanks for flagging this issue, we will investigate this issue and get back to you with some updates. Thanks!

Are there any updates about this?
To this day I am still receiving logging in the legacy dataconnector holding me back from being able to delete the dataconnector.
Maybe there needs to be something changed in the parser to exclude the AMA logs in the legacy connector?

Hi @thom2804,
We have updated template to fix this issue , could you please custom deploy the Solution MainTemplate.
Sharing the MainTemplate and Custom deployment steps-
Custom Deployment - CustomDeploymentSteps.docx
mainTemplate - mainTemplate.json
After custom deploy the mainTemplate ,you will see legacy data connector showing as disconnected.
Then follow below steps to delete this data connector.

Search cisco ASA Legacy dataconnector in Data Connector section.

Click on delete button to delete data connector.

Please confirm delete , after that legacy data connector will be deleted.

Please let us know that your issue resolved or not.
Thanks

Hi @v-rusraut thanks for the quick response, I performed the steps and after doing so the solution showed that the legacy connector is deleted and that I can delete it successfully.
The AMA connector was still active along with the DCR rule I created for this.
Would this same method by removing and reinstalling the solution also work for other solutions which suffer from the same issue? (for example Fortinet)


Getting PI_ERROR_OUT_OF_RESOURCES

Using merged from https://github.com/intel-analytics/ipex-llm/pull/10558 , I retried the 2 GPU execution on an ATSM1 x8 cards system (specs listed here -- https://wiki.ith.intel.com/display/MediaWiki/Flex-170x8+(Inspur+-+ICX)+Qualification)
git cloned - https://github.com/intel-analytics/ipex-llm
I ran the default script under   ../ipex-llm/python/llm/example/GPU/Deepspeed-AutoTP
(run_llama2_70b_pvc_1550_1_card.sh)
export ZE_AFFINITY_MASK="0,1" # specify the used GPU
NUM_GPUS=2 # number of used GPU
export MASTER_ADDR=<IP_ADDRESS>
export FI_PROVIDER=tcp
export CCL_ATL_TRANSPORT=ofi
export CCL_ZE_IPC_EXCHANGE=sockets
export LD_PRELOAD=${LD_PRELOAD}:${CONDA_PREFIX}/lib/libtcmalloc.so:${LD_PRELOAD}
basekit_root=/opt/intel/oneapi
source $basekit_root/setvars.sh --force
source $basekit_root/ccl/latest/env/vars.sh --force
export OMP_NUM_THREADS=$((56/$NUM_GPUS))
export SYCL_PI_LEVEL_ZERO_USE_IMMEDIATE_COMMANDLISTS=2
export TORCH_LLM_ALLREDUCE=1
mpirun -np $NUM_GPUS --prepend-rank
python deepspeed_autotp.py --repo-id-or-model-path 'meta-llama/Llama-2-70b-chat-hf' --low-bit 'sym_int4'
Execution completed with this error --

Hi, memory size of one Flex 170 is only 16GB, therefore, it's quite reasonable that you encountered PI_ERROR_OUT_OF_RESOURCES (which means out of memory) when tried to use two Flex170 to run Llama-2-70b-chat-hf.
Considering the memory size, you could try to run this script(https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/example/GPU/Deepspeed-AutoTP/run_vicuna_33b_arc_2_card.sh) to start instead.

Hi, thank you for the guidance. I tried the recommended script. This is a screenshot of the execution--

Is the bad termination an expected behavior for now?

On another note, the original intent was to try the 2-gpu environment with llama2-7b-chat-hf, i tried to swap the model in the run_vicuna_33b_arc_2_card_mod.sh script --

but getting an error during execution, would you know how i can resolve the error?

Thank you for this guidance. Please see my updates and further comments in the ticket.
Note, will be setting up a debug call and sending an invite shortly.
Regards,
Glen
From: binbin Deng @.>
Sent: Sunday, March 31, 2024 8:11 PM
To: intel-analytics/ipex-llm @.>
Cc: Bertulfo, Glen D @.>; Author @.>
Subject: Re: [intel-analytics/ipex-llm] Getting PI_ERROR_OUT_OF_RESOURCES (Issue #10597)
Hi, memory size of one Flex 170 is only 16GB, therefore, it's quite reasonable that you encountered PI_ERROR_OUT_OF_RESOURCES (which means out of memory) when tried to use two Flex170 to run Llama-2-70b-chat-hf.
Considering the memory size, you could try to run this script(https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/example/GPU/Deepspeed-AutoTP/run_vicuna_33b_arc_2_card.sh) to start instead.
—
Reply to this email directly, view it on GitHubhttps://github.com/intel-analytics/ipex-llm/issues/10597#issuecomment-2029082091, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ARRTRFEGNKM36TUYMKMAW4DY3DF37AVCNFSM6AAAAABFOUVWXSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRZGA4DEMBZGE.
You are receiving this because you authored the thread.Message ID<EMAIL_ADDRESS>

Is the bad termination an expected behavior for now?

Yes, please refer to here for details about the known issue

This issue is resolved with some adjustments

Using this script -- run_vicuna_33b_arc_2_card.sh

**Due to system constraints, this script was recommended for flex executions. Please see previous comments by dev.

For Latency calculations -- I applied recommended BKM from https://github.com/intel-analytics/ipex-llm/blob/main/python/llm/dev/benchmark/README.md

Issue is resolved. Closing this ticket. Thank you team for your help.


GC crashes when compiling and profiling

Running the compiler with the sampling profiler enabled causes intermittent GC crashes.

The crash happens when applying the gc_workhorse with aging_policy while compiling:

N6factor12gc_workhorseINS_11aging_spaceENS_14nursery_policyEEE
data roots
bignum roots
callback roots
literal table roots
sample thread roots
sample callstack roots
size 17299279
N6factor12gc_workhorseINS_11aging_spaceENS_12aging_policyEEE
data roots
bignum roots
callback roots
literal table roots
sample thread roots
sample callstack roots
size 17299279

With a vm compiled with DEBUG=1, you get assertion "(cell)found_block->entry_point() <= address && address - (cell)found_block->entry_point() < found_block->size()" failed: file "vm/code_heap.cpp", line 100, which probably means that the leaf return address used by the signal handler wasn't in the map.


cfndsl_examples

I have created a set of examples at:
https://github.com/neillturner/cfndsl_examples
using cfn2dsl
from Amazon AWS Template Snippets
http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/CHAP_TemplateQuickRef.html

@neillturner These are great, just thinking of the best way of incorporating these.. @howech shall we put them in the examples folder?

I am not opposed to either approach (adding a link in the dox, or adding
them to the examples).
On Wed, May 27, 2015 at 3:06 PM, Neill Turner<EMAIL_ADDRESS>wrote:

Don't you can always point to them in your docs.
—
Reply to this email directly or view it on GitHub
https://github.com/stevenjack/cfndsl/issues/83#issuecomment-106057449.

I'd like to see these sample files being used as part of a cfndsl test suite. The original json template snippets have many un-resolved references and would need to be tidied up so that cfn2dsl can generate files that can be used by cfndsl to 're-generate' the original json.

I have expanded https://github.com/neillturner/cfndsl_examples with more examples and organised into directories. A little disappointed that google 'cfndsl examples' doesn't show my link. if more people link to it, it will go up the rankings.

@neillturner sorry about the delay on this, I think @yumpy has a good point of these being run as part of the test suite.. I think it makes more sense for them to be part of the repo then so we can enhance the the test suite.
We can add in guidelines for when people add new fields to types that they also need to be added to the corresponding examples, would do you think?


Update Docker version to fix ESM error

Resolve #30

Hi @tomjohnburton thanks for your contribution!
Minor thing before merging, can you add a description in the unreleased section of the CHANGELOG located in the docs folder?
Also, did you test this? If yes how?
@DanielMagro97

Hi @dillu24,
Apologies for slow reply. I deployed this Docker image to our own k8s deployment and everything worked as expected.
No other tests were done.


feat: add server-nas

Closes: #173

:tada: This PR is included in version 1.17.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Towards #9: Document Microsoft App Center adapter

The https://in.appcenter.ms/logs endpoint is for the Visual Studio App Center Diagnostics feature.
Microsoft has documentation on how to manually POST to that endpoint: https://learn.microsoft.com/en-us/appcenter/diagnostics/upload-crashes (archived)
There is even a Swagger file: https://in.appcenter.ms/preview/swagger.json (archived)
Additionally, there is also a "Data Collected by App Center SDKs" page: https://learn.microsoft.com/en-us/appcenter/sdk/data-collected (archived)

The https://in.appcenter.ms/logs endpoint is for the Visual Studio App Center Diagnostics feature.

Correction: I'm actually pretty sure now that the endpoint is used for various App Center services, not just Diagnostics.
According to https://learn.microsoft.com/en-us/appcenter/diagnostics/upload-crashes, for Diagnostics, the body should have type: *Error and exception: ….
However, many of the requests we have for the endpoint don't have those properties. For example, https://data.tweasel.org/data/requests/monkey-july-2023,84119 has type: event.


Warm start annotations ignored

I just copy pasted it in the minizinc ide (version 2.2.3) with geocode 6.1.0.
I'm using Windows 10.4.
Here is the output:
`Warning,` ignored search annotation: warm_start_array([warm_start([[xi(1), xi(2)], [i(8), i(4)]]), warm_start([[xf(0), xf(2)], [f(5.6), f(4.7)]]), warm_start([[xs(0), xs(1), xs(2)], [s(), s()]])])
Warning, ignored search annotation: warm_start([[xb(0)], [b(true)]])
x = array1d(1..3, [0, 0, 0]);
xf = array1d(1..3, [1.4, 0.0, 0.0]);
b = true;
xs = array1d(1..3, [{}, {}, 5..9]);
----------
x = array1d(1..3, [0, 0, 1]);
xf = array1d(1..3, [2.4, 0.0, 0.0]);
b = false;
xs = array1d(1..3, [{}, {}, 5..9]);
----------
==========
Finished in 172msec

Are you using gecode or or-tools ?

Geocode 6.1.0.  I got this answer in Stackoverflow (https://stackoverflow.com/questions/60780810/how-to-use-warm-starts-in-minizinc/60795087#60795087)
Basically, the warm_start annotation isnt supported yet with Geocode. They really should add a note about that in the documentation.

This is an or-tools forum, not a gecode one.
Le lun. 23 mars 2020 à 04:15, mvnarvaezt<EMAIL_ADDRESS>a
écrit :

Geocode 6.1.0. I got this answer in Stackoverflow (
https://stackoverflow.com/questions/60780810/how-to-use-warm-starts-in-minizinc/60795087#60795087
)
Basically, the warm_start annotation isnt supported yet with Geocode. They
really should add a note about that in the documentation.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/google/or-tools/issues/1931#issuecomment-602362891,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACUPL3MFLDOYMYXEADWBTDTRI3H3HANCNFSM4LQTO6SA
.


Contextual menu icons could be bigger

Hello!
I wanted to ask if it's possible to scale up a bit the size of the icons in the context menus.
Right now they are very little and the image is very difficult to perceive even on a close monitor due the big contrast with the background.
Please refer to the attached screenshot below.
Describe the solution you'd like
To enhance readability, I suggest making the icons slightly larger and adjusting the contrast between the icons and the dark menu background. This change would also improve the visibility of disabled items, making the overall interface easier to navigate.
Thank you!

This is something weird! Not sure when this happened, the icons should be like this. The image is from one of the old tutorials

And not like this

I wonder if we made some change or Windows 11 is doing something for us 🤣

Well at least I am glad it's not the intended size and something is wrong somewhere :D anyway yeah I am on windows 11 and I am at 100% DPI settings on all monitors.

I don't have time to have a closer look at this, but the first that comes to mind is #1807, there's a bunch of width/height value changes that may be the cause of this issue.

Yeah if you remove the Margin="5,5" from MenuItem ControlTemplate ContentPresenter in Stride.Core.Presentation.Wpf\Themes\ThemeSelector.xaml the icons can stretch to the max of their parent:

En passant I see that the spacing system is kinda mixed, i see lot of multiple-of-5 margins which are not great for application scaling.