Fix TimeStamp kMaxSeconds and kMinSeconds

Summary:
The comment in timestamp says that
" Presto's Timestamp is stored in one 64-bit signed integer for
milliseconds, this range ensures that Timestamp's range in Velox will not
be smaller than Presto"
the -1 should be for kMaxSeconds not for kMinSeconds, tests updated.
Differential Revision: D49794381

@liujiayi771 Would you please help take a look?

The maximum Presto's Timestamp in Velox should be
Timestamp(seconds=int64_max/1'000, nanos=(int64_max % 1000) * 1'000'000).
So I think the original range is correct.
In the case of negative Timestamp, because the integer is rounded upward when divided by 1000, we need to subtract 1 to correct the effect of rounding. The minimum Presto's Timestamp in Velox should be
Timestamp(seconds=int64_min/1'000 - 1, nanos=(int64_min % 1'000 + 1'000) * 1'000'000).
@mbasmanova We've discussed this before.
https://github.com/facebookincubator/velox/pull/5083#issuecomment-1584644878

@liujiayi771
as the diff summary says "when converted to milli the time stamp should fit in int64_t" which means toMilli() should not
overflow, you can try TimeStamp(maxSec, maxNanos) and see that it does overflow currently.
INT64_t_MAX =<PHONE_NUMBER>854775807
INT64_t_MAX/1000 =<PHONE_NUMBER>854775
MaxNano  =  999'999'999;
if we construct TimeStamp(9223372036854775,   999'999'999)
when we convert to milli we do
miilis =<PHONE_NUMBER>8547751000 + 999'999'999/1,000,000;
miilis =<PHONE_NUMBER>8547751000 + 999'999'999/1,000,000;
=<PHONE_NUMBER>854775000 + 999 = -9223372036854775617 OVERFLOW
you can try the same thing on the new numbers to see why they make sense
thanks for taking a look!
another easy way to check it to make sure toMillis() does not throw for max and min time stamps.
right now it does.

@laithsakka
I understand that the main purpose of this modification is to avoid overflow when the toMilli() method is called on the min/max Timestamp in Velox.
But this will make the range of Timestamp in Velox smaller than Presto. This modification will make the milliseconds range of Timestamp be [-9223372036854775000,<PHONE_NUMBER>854774999], and the range of Presto will be [-9223372036854775808,<PHONE_NUMBER>854775807], which may cause inconsistencies with Presto at boundary values.
But this range itself doesn't mean much, it's just a custom limit. From the perspective that the toMilli() method will not overflow, the range you modified is correct, but from the perspective of the range, it may cause some problems. The fundamental reason here is that the way Presto and Velox express Timestamp is inconsistent, and it is difficult to completely unify them.

@liujiayi771
I see this sounds like a bug in presto, since those timestamps in presto wont be shuffled.
also if we decided to keep the ranges as is we might want to remove the comment that a time stamp in presto fit in int64_t
I started a related discussion in presto
https://github.com/prestodb/presto/issues/21025
@mbasmanova  any thoughts on this?

@liujiayi771 I see this sounds like a bug in presto, since those timestamps in presto wont be shuffled. also if we decided to keep the ranges as is we might want to remove the comment that a time stamp in presto fit in int64_t
I started a related discussion in presto prestodb/presto#21025
@mbasmanova any thoughts on this?

It seems that fromUnixTime function in Presto should return int64_max instead of using Math.round and casting to long when input double value is too large.

@laithsakka Laith, I took a look and shared my findings in https://github.com/prestodb/presto/issues/21025#issuecomment-1748556485
The existing test cases appear correct to me. The modifications appear incorrect.

Thanks you guys for looking at this
two questions

why are we limiting the seconds if anyway we do not gurantee that the represented time fit in milli?

Still do not understand why the min seconds is
std::numeric_limits<int64_t>::min() / kMillisecondsInSecond - 1
and not
std::numeric_limits<int64_t>::min() / kMillisecondsInSecond
@ma

ahh i see the answer for (2) now

@laithsakka
At first we just wanted to fix the problem with the Timestamp::toString method, and we thought that too large seconds would not make much sense in actual use. You can check this discussion https://github.com/facebookincubator/velox/pull/5083#discussion_r1222511606.
Later we thought we could choose the same range as Presto, but since Velox uses two integers(seconds and nanos) to represent Timestamp, which is not completely consistent with Presto. So we choose a range that is as close as possible, and it needs to be larger than the range of Presto.
If Velox need to be completely consistent with Presto, we need to limit the value of nanos at the boundary. For example, when seconds=kMax, nanos <= kMax % kMillisecondsInSecond * kNanosecondsInMillisecond. However, we previously thought that doing so was not necessary and would make the value of nanos becomes complicated, we just want a roughly reasonable range.


chore: cleanup spelling syntax and typos

Just a PR to cleanup some typos and improve syntax in a few areas. Some of these had been resolved previously, but got dropped somewhere before we formalized our branches/release process.

I know its annoying and trivial, but it would be nice if we also followed idiomatic JS naming where we can. (Noticing lots of snake_case vars here and there)

I hear you. This was carried over from the openAI docs, which tends to use snake case for function tool names, which we then map to the idiomatic javascript function names. Will push up changes.

Figured as much 👍 just trying to get ahead of the point where we start promoting code and / or incorporating. Its definitely no big deal.


Backend refactor and small fixes

Sorry for the wholesale PR.
Commit messages contain the rationale for each change and they should be independently cherry-pickable, modulo b7971c6.

399c787 fixes #28
fb5e4e0 fixes #27
31bc7b5 fixes #24

These are good fixes. I'm going to merge non-backwards-compat in this weekend and take them with it.


Provider logic + Controllers

Summary

Added provider initialization
Added controllers based on web3modal implementation
Added basic view to show user address + disconnect button

Screenshots
https://user-images.githubusercontent.com/25931366/229607879-c122469b-f58c-403a-bab5-cc08674c7b3d.MP4

Thanks for your feedback @0xAsimetriq 🎉
About image sizes, im following react-native docs recommendations, and i think there's no maintenance problem because Figma has the option to export in all sizes.

I left few comments on code style, that apply all around. I think it might be time to introduce tools like eslint / prettier to catch them? Also perhaps set up github actions to run these checks on pr's (refference in web repo).

+1 @0xAsimetriq so let's add some GHub Actions for this. Maybe also let's consider leaving an Expo Snack as our version of Vercel. Then we can easily scan without having to do the xCode / RN CLI setup each time? Do you know of any other streamlined flows @ignaciosantise ?

NonBlocking: Let's also add a README to the main directory and for examples.

Running into this while doing examples. I did yarn and normal Expo scan/ login. Added projectID. Anything I am missing?

NonBlocking: when I did a yarn on the examples repo, it asked me to ✔ It looks like you're trying to use TypeScript but don't have the required dependencies installed. Would you like to install<EMAIL_ADDRESS><EMAIL_ADDRESS>… yes
Is this something we need to add or is it more an expo thing?

@crypblizz8 in order to run the example go to the root folder and:

Run yarn to install sdk and example dependencies
Run yarn example ios or yarn example android

 Running into this while doing examples. I did yarn and normal Expo scan/ login. Added projectID. Anything I am missing?

🤔 could you run yarn jest --clearCache in root folder? and try again

NonBlocking: when I did a yarn on the examples repo, it asked me to ✔ It looks like you're trying to use TypeScript but don't have the required dependencies installed. Would you like to install<EMAIL_ADDRESS><EMAIL_ADDRESS>… yes
Is this something we need to add or is it more an expo thing?

To run this example go to the root folder and:

Run yarn to install all deps
Run yarn example android or yarn example ios


Intel Syntax does not work for gcc

GCC compiler output shows operands in AT&T sequence, even when Intel output is selected.  clang looks ok.
Case in point:
https://www.godbolt.org/z/rpujEP

This is unfortunately just the way it is with assembly output in combination with manual asm in code. There have been other similar issues raised.
I’m not sure if there’s a way to “fix” this.
The 11010 output will be without this inconsistency, of course.
Sent from my iPhone

On 15 Nov 2018, at 17:41, bkin<EMAIL_ADDRESS>wrote:
GCC compiler output shows operands in AT&T sequence, even when Intel output is selected. clang looks ok.
Case in point:
https://www.godbolt.org/z/rpujEP
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

As @partouf says, this is not a Compiler Explorer issue as such. The underlying GCC outputs the asm block as text, without attempting to parse it. Clang, on the other hand, parses the asm and re-outputs it.
Consider this: https://www.godbolt.org/z/ZLRWL_ for example. I'm going to close this as I don't have a good solution for this that doesn't involve compiler hackery.

Ok, I understand, thanks for the explanation. I thought that the assembler output works by 'objdump -d'ing the object files, hence I was worried about a miscompilation.


st2 webhook get -h now asks for user to provide a webhook than ref-or-id

Closes: https://github.com/StackStorm/st2/issues/4048
Related: https://github.com/StackStorm/st2/issues/4070
((virtualenv) root@80fa4dcc1299:/st2# st2 webhook list
+----------------------------+------------------+-------------+
| url                        | type             | description |
+----------------------------+------------------+-------------+
| devices/config             | core.st2.webhook |             |
| ping_validation            | core.st2.webhook |             |
| remove_operator_limit_port | core.st2.webhook |             |
| restart_radius             | core.st2.webhook |             |
+----------------------------+------------------+-------------+
(virtualenv) root@80fa4dcc1299:/st2# st2 webhook get restart_radius
+------------+--------------------------------------------------------------+
| Property   | Value                                                        |
+------------+--------------------------------------------------------------+
| type       | core.st2.webhook                                             |
| id         | 5ac79a8b652cf12d48d02aad                                     |
| name       | d4628f21-12db-4a59-8789-f5d59a387d8f                         |
| pack       | core                                                         |
| parameters | {                                                            |
|            |     "url": "restart_radius"                                  |
|            | }                                                            |
| ref        | core.d4628f21-12db-4a59-8789-f5d59a387d8f                    |
| uid        | trigger:core:d4628f21-12db-                                  |
|            | 4a59-8789-f5d59a387d8f:e107a2c548b03bc275b4e6a1c2c1e08c      |
+------------+--------------------------------------------------------------+
(virtualenv) root@80fa4dcc1299:/st2# st2 webhook get -h
usage: st2 webhook get [-h] [-t TOKEN] [--api-key API_KEY] [-j] [-y]
                       [-a ATTR [ATTR ...]]
                       url

Get individual webhook.

positional arguments:
  url                   Url of the webhook.

optional arguments:
  -h, --help            show this help message and exit
  -t TOKEN, --token TOKEN
                        Access token for user authentication. Get
                        ST2_AUTH_TOKEN from the environment variables by
                        default.
  --api-key API_KEY     Api Key for user authentication. Get ST2_API_KEY from
                        the environment variables by default.
  -j, --json            Print output in JSON format.
  -y, --yaml            Print output in YAML format.
  -a ATTR [ATTR ...], --attr ATTR [ATTR ...]
                        List of attributes to include in the output. "all" or
                        unspecified will return all attributes.
(virtualenv) root@80fa4dcc1299:/st2#

TODO

[X] Fix API documentation

How do we handle leading slashes and such in the name / url?
I remember us having some tests for that so we should probably be fine.

How do we handle leading slashes and such in the name / url?
Doesn't work today. It used to. Opened https://github.com/StackStorm/st2/issues/4070.

I need to include some other changes in v2.7.0 so I will merge this into master and also add it to a PR which will include some other changes we want in v2.7.0.


Carthage framework project/files not updated to Swift 4.2

Carthage build fails because the files in the framework inside the FRadioPlayer.xcodeproj are not updated to Swift 4.2.

Edit: The code is updated to Swift 4.2, but the framework target is still set to Swift 4 and will need to be updated to Swift 4.2. Deployment target is also required to be updated to at least iOS 10.0 for it to build without errors. That should be a quick fix. Thanks in advance!

Hey @fahlout , thanks for reporting that I will fix the issue ;)

@fahlout I fixed the issue, update your Cartfile to:
github "fethica/FRadioPlayer" ~> 0.1.12

Thanks!


some Chinese characters resulted in CID notation

Hi euske,
I'm dealing with chinese characters recently and I've properly installed CJK support as instructed.
For some PDF files pdfminer can deliver correct chinese results but others just don't work and keep printing out notations like (cid:1117). Below are some examples:
PDF with correct results
PDF with CID results
Would you kindly point out the difference between those 2 pdf?
Thanks

Hi, has this problem been solved and how?


adding in blog post announcing conference scholarships

Awesome. Adding tonight!

On Mar 2, 2016, at 5:39 PM, Nell Shamrell-Harrington<EMAIL_ADDRESS>wrote:
You can view, comment on, or merge this pull request online at:
https://github.com/OperationCode/operationcode/pull/113
Commit Summary
adding in blog post announcing conference scholarships
File Changes
A blog/_posts/2016-03-02-conference-scholarships.markdown (24)
Patch Links:
https://github.com/OperationCode/operationcode/pull/113.patch
https://github.com/OperationCode/operationcode/pull/113.diff
—
Reply to this email directly or view it on GitHub.

:)


Peers for Gateway

Signed-off-by: Josh Horton<EMAIL_ADDRESS>Final review of Peers page/topic for Fabric Gateway GA
Type of change

New feature
Documentation update

Description
Fabric Gateway service - new model for peers and apps for transaction proposals and endorsements.
Related issues
https://github.com/hyperledger/fabric/issues/2807

Calling with @andrew-coleman to avoid more cycling on GitHub PRs.
Closing to do a new one to follow.

Replaced by https://github.com/hyperledger/fabric/pull/3091


[PE-D][Tester D] Inaccurate Error Message for Editing RECIPE_DURATION

When I input an invalid value for RECIPE_DURATION for edit, the error message displayed is telling me that I have to input at least 1 field to edit, which I already did. Would a better message be to tell me that the input is incorrect?

Labels: type.FeatureFlaw severity.Medium
original: eugenetangkj/ped#4

Fixed in #245
negative numbers are also not allowed for portion/duration ranges (will not match the regex), so the proper error message is thrown in that case too


Found New Error Message

I am seeing Error 403 Forbidden message. Can you think of something to help here. Please take a look at screenshot.
Thank you,

Probably wrong certificate, the certificate is what is used as trust so basically it should be the only thing causing a 403. Did you export the client certificate from QMC, then importing it into the Certificate Store?

Rikard, thanks for your reply.
I am testing this on the same machine i.e. QS and IIS are on the same machine. I have not exported the client certificate because the certificate got installed in the Certificate Store automatically after QS installation. Here is the screenshot. Do you think I still need to export certificate?
Thank you,

From: Rikard Braathen<EMAIL_ADDRESS>Sent: Monday, January 9, 2017 7:50 AM
To: braathen/qlik-auth-net
Cc: sohailansari2; Author
Subject: Re: [braathen/qlik-auth-net] Found New Error Message (#3)
Probably wrong certificate, the certificate is what is used as trust so basically it should be the only thing causing a 403. Did you export the client certificate from QMC, then importing it into the Certificate Store?

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/braathen/qlik-auth-net/issues/3#issuecomment-271288691, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVSzTrkOyEH4WicdpisHsRM09OE8r3zhks5rQjsugaJpZM4LdefU.

Yes, try it... :) or look for the certificate called QlikClient.

I have exported the Certificate out of QMC and then imported it into MMC as you have described in one of your PDF document. That created a duplicate certificate (screenshot attached).
The error message is not different as it says "Could not create SSL/TLS secure channel."(screenshot attached).
I am unable to find any certificate by the name of QlikClient. I have read that QlikClient certificate has been discontinued with QS version 3.0
Thanks,

From: Rikard Braathen<EMAIL_ADDRESS>Sent: Monday, January 9, 2017 8:32 AM
To: braathen/qlik-auth-net
Cc: sohailansari2; Author
Subject: Re: [braathen/qlik-auth-net] Found New Error Message (#3)
Yes, try it... :) or look for the certificate called QlikClient.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/braathen/qlik-auth-net/issues/3#issuecomment-271298257, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVSzTlpcJxfCmLtSNs78qOOIwXHzkdgjks5rQkTugaJpZM4LdefU.

Rikard, I got it working now. Thanks for your advice. It was in fact "QlikClient" certificate that was missing. The "QlikClient" certificate got imported into MMC when I imported the "Client" certificate. And your code started working.
However, I had to make a minor change in your code and recompile it to make it work i.e.
On QlikAuthNet.cs file, I had to change line number 43
CertificateLocation = StoreLocation.CurrentUser;
to
CertificateLocation = StoreLocation.LocalMachine;
Thank you for all your help.
Regards,
Sohail Ansari

From: Rikard Braathen<EMAIL_ADDRESS>Sent: Monday, January 9, 2017 8:32 AM
To: braathen/qlik-auth-net
Cc: sohailansari2; Author
Subject: Re: [braathen/qlik-auth-net] Found New Error Message (#3)
Yes, try it... :) or look for the certificate called QlikClient.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/braathen/qlik-auth-net/issues/3#issuecomment-271298257, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVSzTlpcJxfCmLtSNs78qOOIwXHzkdgjks5rQkTugaJpZM4LdefU.

I was under the impression the CertificateLocation was configurable as options, maybe I need to have a look at that some day, it was contributed by someone a while ago. Anyway, glad that you got it working, good job! Closing this now...


feat: ✨ syn-side-nav in rail mode should open automatically when a syn-nav-item is focused in rail mode

Checklist (for the issue creator)

[x] I have looked at the documentation site and was not able to find the required feature.
[x] I have looked into the Figma boards and was not able to find the required feature.
[x] I would be willing to provide a PR for this feature.

Component
I suggest to add the feature to the following component: <syn-side-nav>
Story

As a User of an app built using the Synergy Design System, I would like the <syn-side-nav> to automatically expand when it is in rail mode and I navigate via keyboard interaction so that I can see the item that is currently active.
Acceptance criteria

[ ] All DoD items successfully completed
[ ] <syn-side-nav> should automatically open when a slotted <syn-nav-item> is focused via keyboard interaction and rail mode is active.
[ ] <syn-side-nav> should automatically close when no <syn-nav-item> is selected, rail mode is active and the user is not currently hovering over the navigation.

As discussed in https://github.com/synergy-design-system/synergy-design-system/pull/466#discussion_r1607812562, we also have to remove the current focus styling on the nav-item. The change suggested in the PR linked must be done here instead (e.g. remove  .nav-item--horizontal:focus .current-indicator--visible)


Implementation of the result and badge module

Feat result : The "result" API allows us to obtain all the information in a repository
Feat badge : The "badge" API, allows us to hold the badge of a repository
Feat buildURL : The buildURL method, allows the construction of a valid url

If the module is for Node usage only, maybe should we consider using undici then we could write tests easily for methods in index.ts


Enable Linear Models By Default

From what I can tell, linear models are not run by default in version 0.3.1. What is the reason? Would you consider adding them to the default hyperparameters file?
To run these models do I just use the hyperparameters argument?
predictor.fit(..., hyperparameters={'LR': {}})

Thanks for making this issue!
Yes you can use linear/logistic regression in the exact way you mention.
Linear models are currently not enabled by default as I haven't gotten around to testing their impact and stability, as the primary difficulty is that they don't seem to have a way to stop the training process via callbacks (at least from what is currently coded). What that means is it may take much longer than the specified time limit to train. This requires a bit of work to make sure it works smoothly on large datasets, but is definitely high priority for 0.4 release as it should in most cases be very fast to train.

Update: I've done some testing and adding linear models does indeed enhance Tabular's performance across our benchmark, but I need additional time to sanity check the user experience impact from adding linear models.


Assign judgment to self from index page

TODO: say who judgment is assigned to on search page.
TODO: pull bugfixes into separate commit.
Needs some UI love.
Changes in this PR:
Trello card / Rollbar error (etc)
Screenshots of UI changes:
Before
After

[ ] Requires env variable(s) to be updated

 👩💻EUI - Assign Judgments HTTP endpoint


Setup release protocol

AC:

Ensure Dev is up to date with master, or cherry pick separate branches from dev for different PRs in future
Delete dev branch
Setup honesty-store-kiosk-QA site
Setup private branches env-QA and env-Prod
Setup automatic deployment to respective sites for branches
Change master automatic deployment to point to honesty-store-kiosk-dev
...

The env-qa branch needs protection added to it + myself and @KyleADay to be set as 'code owners' on the branch

@eclennett Set up code owner protection on the env-qa branch


how can i use active for menu-iten?

is it a css or an attribute?
how can i indicate that md-menu-item is activated?
  <md-menu #menuModules="mdMenu"  y-position="below" x-position="before" >
            <button md-menu-item *ngFor="let repo of parent.repos" (click)="parent.navigate('github/' + repo.name + '/README.md' )" >
                {{ repo.description }}
            </button>
        </md-menu>
html

Github issues are for bugs and feature requests, use stackoverflow or gitter for support requests.


Update nf-tlhelp32-module32firstw.md

Fix typos regarding MODULEENTRY32W.

@k15tfu : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Duplicate violation message on the same resource

terrascan version: 1.10.0

Description
I am receiving duplicate violations when running terrascan against a Github resource. The violation in question is accurics.github.IAM.1 (AC_GITHUB_0002). When I put in the skip in the resource to ignore the violation (#ts:skip=AC_GITHUB_0002), it only skips the one violation and not both
What I Did

Create resource "github_repository" and set visibility as a variable
Run Terrascan which reports the same violation twice on the resource (accurics.github.IAM.1)
Add the skip to ignore the violation (#ts:skip=AC_GITHUB_0002)
Run Terrascan again which skips one violation but not both

I have also tried duplicating the skip command and it didn't do anything.

Hi @Official-James, do you have an example we can use for replicating the issue?

To replicate:
Initial Terraform Files
filename: main.tf
resource "github_repository" "resource_name" {
  name        = var.name
  description = var.description
  visibility = var.visibility

filename: var.tf
variable "name" {
  type        = string
  description = "The name of the repository."
}

variable "description" {
  type        = string
  description = "A description of the repository."
}

variable "visibility" {
  type        = string
  description = "A description of the visibility"
  default     = "private"
}

Results:
Violation Details -

	Description    :	Repository is Not Private.
	File           :	main.tf
	Module Name    :	root
	Line           :	1
	Severity       :	MEDIUM

	-----------------------------------------------------------------------

	Description    :	Repository is Not Private.
	File           :	main.tf
	Module Name    :	root
	Line           :	1
	Severity       :	MEDIUM

	-----------------------------------------------------------------------

Scan Summary -

	File/Folder         :	/main.tf
	IaC Type            :	terraform
	Scanned At          :
	Policies Validated  :	2
	Violated Policies   :	2
	Low                 :	0
	Medium              :	2
	High                :	0

Add skip to violation
filename: main.tf
resource "github_repository" "resource_name" {
  #ts:skip=AC_GITHUB_0002
  name        = var.reponame
  description = var.description
  visibility = var.repovisbility

Result of skip
Violation Details -

	Description    :	Repository is Not Private.
	File           :	main.tf
	Module Name    :	root
	Line           :	1
	Severity       :	MEDIUM

	-----------------------------------------------------------------------

Skipped Violations -

	Description    :	Repository is Not Private.
	File           :	main.tf
	Module Name    :	root
	Line           :	1
	Severity       :	MEDIUM

	-----------------------------------------------------------------------

Scan Summary -

	File/Folder         :	/main.tf
	IaC Type            :	terraform
	Scanned At          :
	Policies Validated  :	2
	Violated Policies   :	1
	Low                 :	0
	Medium              :	1
	High                :	0

Add skip again produces the same above result.


Improve speed of file explorer node expansion and collapse

The tree feels smoochy. This is likely a setting n the library. Making it expand and collapse without delay would be much better IMHO

Are you talking about the expansion and collapse of nodes in the node view? Or are you talking about expansion and collapse of the directories and files in the left tree pane?

@jdaguil The left tree pane, explorer or finder-like with files and directories. Side note: I really like that this is keyboard friendly btw :+1:
.

@pombredanne can you quantify with some example times? We previously had it load everything at once, but then there was a delay before the tree showed up after loading a file. We switched to lazy loading so that the tree would show up immediately regardless of how big the SQLite file is. This made load times faster, and we could also save ram for large databases. Likely, if it's taking a long time it's because there's a lot of files in the tree, so loading them all in memory would increase ram usage and slow down the initial SQLite load time.

This is not a matter of lazy loading IMHO, but rather some setting on the tree UI lib that does "smooth" collapse and expand, as some sorts of springy visual effect trying to be nice, but this is not nice IMHO. There is some delay or effect that can likely be disabled.

I've just pushed a fix for what I think you are referencing. I misunderstood what you were saying until the last iteration.


SteamAudio crashes Unity on stopping game in Editor

System Information
Please provide the following information about your system:

Steam Audio version: 4.5.0-4.5.3 (previous versions not tested)
(If applicable) Unity version: 2021-6000 (many sub versions between tested)
Operating System and version: Windows 10/11 (many sub-versions tested)
(Optional) CPU architecture (e.g. x86-64, armv7): x64 (AMD and Intel tested)

Issue Description
We wanted to use Steam Audio in our projects but in every project on every tested Unity and SA versions on many different computers we got crashes. Always on stopping game in editor (not pause). I tried to fix it as task in work but no luck.
It's crashing randomly, sometimes whole day is fine, another day it crashes on every stop. One day it crashes for one person but not for the other (on the same project version). It's not crashing is there is no audio scene, I mean no audio scene baked, and no probes etc. For one of our projects we made optimized Unity raycaster for occlusion/transmission and we were using only this and HRTF and nothing else and then it was completely fine, but when we bake scenes, bake probes etc and use them then lottery begins.
In Android builds it's fine.
Technical info:
It's always crashing on _aligned_free in Memory::free (memory_allocator) with Heap Corruption Exception.
Mostly from struct deleter (same file as above) but sometimes from other sources too (like CSource::release invoked from API from Unity).
I even added list with all addresses allocated with Memory class and on every Memory::free I'm checking if this address is allocated and if yes only then I'm doing _aligned_free and removing this address from list and it still crashes.
It's crashing on normal version from GitHub I just attempted to fix this.
Crash Dump
I will add it soon because dumps I have now are from my modified version.

We've been having similar issues since we integrated FMOD + Steam Audio into our project. Are you using reflections at all? We've had significantly less crashes on runtime end since we disabled that, but that's purely anecdotal.

We aren't using FMOD and don't know about reflection (if we use them) but even if, we don't use them on Steam Audio.
We had this problem on at least 4 different projects (and on different Unity versions) and one of them was so basic that I'm sure there was no reflection used.


Please include a LICENSE file

I see the license is MIT I was hoping you could add a LICENSE file. Thanks

I can create a PR if appreciated?

Fixed via TOOLS-2046, commit 674ccf90e7aeb0f8cb04a764822915146133b385.


Cherry-pick #619

feat(snapshot/commitment): use commitment when creating a snapshot

The snapshot create now uses the same rules as volume/replica creation.

Signed-off-by: Tiago Castro<EMAIL_ADDRESS>

ci: fix test race condition

Make cache reload period too high so we don't try to refresh whilst testing
a timeout condition.

Signed-off-by: Tiago Castro<EMAIL_ADDRESS>

bors merge

bors cancel

bors merge

bors cancel

bors merge


Replace the word 'shaded' with 'relocated'

Why are the changes needed?

Follow the discussion https://lists.apache.org/thread/v1qswzoxd6zgpbpw3ddm97hry3g5sn6l to rename the modules' names, also supply the module description.
How was this patch tested?

[ ] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[ ] Run test locally before make a pull request

Can we also add a README?


img2img_allen_brain problem defs

only outstanding issue blocking this should be that the test in allen_brain_test.py that uses tf.eager breaks many/all other t2t tests not compatible with eager mode.
squashed these:
commit 76c8cd3f105c07919f84d70b02b1fbd38b5376fe
Author: cwbeitel<EMAIL_ADDRESS>Date:   Fri Jun 15 18:35:13 2018 +0000
fix mock_raw_data

commit f0dc3a0fcb6691c5b8854f419485966cb2f7b9c2
Merge: 3e17473 4b7360d
Author: cwbeitel<EMAIL_ADDRESS>Date:   Thu Jun 14 22:54:33 2018 +0000
Merge branch 'allen-deps' of https://github.com/cwbeitel/tensor2tensor into allen-deps

commit 3e174736ec6628fe0b8097c1a3b0c2b3646ddee9
Author: cwbeitel<EMAIL_ADDRESS>Date:   Fri Jun 8 17:04:39 2018 +0000
cleanup

commit 3e8ef9d61382afd795a26a16ffb254974a80dc8e
Author: cwbeitel<EMAIL_ADDRESS>Date:   Wed Jun 6 23:07:28 2018 +0000
removed image summary writing from metrics

training was 10x slower than CIFAR img2img problem, profiled and found 91% of time was being sp

ent writing ImageSummary's, removed that from metrics and saw the speed increase as expected.
commit<PHONE_NUMBER>96724da4ab7878eff7d84ed495af44
Author: cwbeitel<EMAIL_ADDRESS>Date:   Tue May 29 16:00:17 2018 +0000
basic img2img prob. defs that maybe get data

commit 320f06c05697e35a108e74e7d3a0fbd29bebee3b
Author: cwbeitel<EMAIL_ADDRESS>Date:   Thu May 24 18:53:02 2018 +0000
test without accessing network, follow rcfile

* made test that accesses remote API optional, off by default
* included test that runs sub-image extractor on mocked raw input image within mocked directory

structure mirroring what would be produced by running actual data downloader
* refactoring according to rcfile
commit 503123b09e1d784f9ca441c4b3f723d99baba7fb
Author: cwbeitel<EMAIL_ADDRESS>Date:   Wed May 23 22:09:31 2018 +0000
add [allen] deps to travis setup

commit 239188b8c3e28d6fb0fb31c0d9167291874dc2f1
Author: cwbeitel<EMAIL_ADDRESS>Date:   Wed May 23 21:39:13 2018 +0000
data download and sub-image gen utility

* if additional [allen] deps not present suggest running `pip install tensor2tensor[allen]`
* obtains a list of section dataset ID's from Allen Institute API
* downloads specified number of raw images from specified number of section datasets
* produces specified number of sub-images of specified size from raw images.
* simple e2e test that runs the data downloader and sub-image generator with default parameters

commit af34cc660c90663b662eaa2bc7a5a66cfadf8762
Author: cwbeitel<EMAIL_ADDRESS>Date:   Wed May 23 17:47:35 2018 +0000
deps. for allen brain img probs. via extras

@rsepassi Noticed many other tests were failing outside of allen_brain* tests with a bunch of errors about Eager mode, guessing the problem for those is my test that uses Eager.

Where are you seeing the errors about Eager mode? I don't see them in the Travis test logs.
When you run the tests locally with pytest, do they pass?
We'll want to ensure these all pass on Travis. One idea is that maybe it's problematic running the allen tests alongside all the others and we'll want to run them separately. You can see in the .travis.yml file examples of a test being ignored in the main run and then run separately afterwards.

Looks like the allen tests are failing.

@rsepassi Those changes appear to have fixed everything except for a linter error in gym_utils, https://travis-ci.org/tensorflow/tensor2tensor/jobs/394214469#L2613,
pylint -j 2 tensor2tensor
Using config file /mnt/nfs-east1-d/work/prs/problem-def/pylintrc
************* Module tensor2tensor.data_generators.gym_utils
E1123: 31 WarmupWrapper.__init__: Unexpected keyword argument 'dtype' in constructor call [unexpected-keyword-arg]
W0221: 48 WarmupWrapper.reset: Parameters differ from overridden 'reset' method [arguments-differ]
E1123: 70 PongWrapper.__init__: Unexpected keyword argument 'dtype' in constructor call [unexpected-keyword-arg]
E1123:149 BreakoutWrapper.__init__: Unexpected keyword argument 'dtype' in constructor call [unexpected-keyword-arg]

Verified the allen_brain* files pass the linter locally.

@rsepassi My bad, there was a little linter fix I made and forgot about. I can make those changes to gym_utils if that simplifies things.

Hmm looks like those may not have actually been part of the problem on travis.

@rsepassi Let me know if you're waiting on me for anything, otherwise no pressure!

Was just going through to merge it but noticed that AllenSDK has a non-permissive license which actually makes it hard to import and use the code within Google. It may still be possible but it would be a lot of work and I just don't have the cycles to go through something like that right now unfortunately.
Now I don't want your work to go to waste so here's what I'm thinking: you can maintain a fork of tensor2tensor with the Allen problems so that people can generate the data and we'd be happy to have the Problems in tensor2tensor just without any AllenSDK dependency/import and the generate_data method could print out "To generate the data for XXX go to github/cwbeitel/tensor2tensor and run ....". Or something like that.
Very sorry about this, wish I noticed it earlier. What do you think?

@rsepassi Hey! I removed the allensdk dependency but kept the data download utils which is far simpler if maintaining a list of image IDs (instead of building these through various queries to the API). There are 100 IDs included here randomly sampled across the full set of Mouse data. Given that we're considering 64 x 64px regions of images that are like 8.5k x 15k that's enough to generate around 3M examples. If that's not enough I referenced the API docs and a gist that includes the old download code.

Thank you @cwbeitel and sorry for the delay. Will merge shortly.

@rsepassi No worries. Thank you.


Week 39: Docker for Java Developers Part 1

Please provide your feedback by posting a comment against this issue.

When you are building all kinds of docker images, you may end up with a lot of untagged images, volumes.  But it's nice to have something to clean up locally.

@soleo thanks. I have added it to the blog.


Converted to Android Studio format

Ran through the Android Studio converter to the new Project Format with Gradle.

Although a conversion to gradle and Android Studio is good, this PR is dangerous because it destroys the commit history of pretty much every file.

@chrisballinger Why's that? Git is pretty good in tracking file moves. See e.g. git log --follow <filename>.
That said, I'm not a fan of switching to Android Studio. Also using some converter for the build system migration is always a potential cause of problems.

Thanks for the thought, but we are working on our own update and conversion at the moment!


Need a clear link to the GitHub project

There seems to be no direct link to the project from the website itself (unless I totally missed it).
I think it's important to have a clear way for external people to come over here to contribute if they want while keeping it easy for non-technical people to interact through the forms (which are awesome btw :grinning: ).

The only link back to Github is on the "Report a problem on this page" button at the bottom of the page and that links to Issues and is only on the homepage.
We should add another button to 'View source code and data" and have it on every page

You can find the project link back to github in "open source code", since the project is added to the catalog.

However we could also add a direct link on all pages.
I would suggest to add it in the dropdown menu or maybe beside it (upper right side perhaps). What do you guys think? Which would be best in terms of design?

I think if we had the buttons that are on the home page on every page, we could do something like

See design guide for buttons
https://wet-boew.github.io/wet-boew-styleguide/design/buttons-en.html#btn-groups


Live Unit Testing and code generation

I'm using live unit testing with Microsoft Orleans, which has a Microsoft.Orleans.CodeGenerator.MSBuild package that generates code at build time.  This doesn't appear to be working with LUT, is there something I need to configure so that LUT runs the package targets during build?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d963b3f7-abc8-a9e7-9ac7-2a5c7ad53c28
Version Independent ID: 2866e780-84b2-a833-24fb-8a4354a9c0f4
Content: Live Unit Testing FAQ - Visual Studio
Content Source: docs/test/live-unit-testing-faq.md
Product: visual-studio-windows
Technology: vs-ide-test
GitHub Login: @Mikejo5000
Microsoft Alias: mikejo

@jbockle -- thank you for your question.  Please consider these resources:

https://social.msdn.microsoft.com/Forums/vstudio
https://www.stackoverflow.com
https://www.reddit.com/r/VisualStudio/

cc @Mikejo5000 for awareness
I am closing this issue now.  You are welcome to @ mention me for any followup.
We hope to hear from you again.


make step extensible to support implementing new protocols and test types for python version

Codecov Report

Merging #1222 (eac548f) into master (8aabf5a) will decrease coverage by 9.54%.
The diff coverage is 57.14%.

@@            Coverage Diff             @@
##           master    #1222      +/-   ##
==========================================
- Coverage   67.88%   58.33%   -9.55%
==========================================
  Files          62       35      -27
  Lines        7352     4587    -2765
==========================================
- Hits         4991     2676    -2315
+ Misses       2147     1696     -451
- Partials      214      215       +1

Flag
Coverage Δ

unittests
58.33% <57.14%> (-9.55%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

hrp/response.go
65.21% <0.00%> (ø)

hrp/step_api.go
53.84% <ø> (+7.57%)
:arrow_up:

hrp/step_request.go
51.57% <25.00%> (-0.49%)
:arrow_down:

hrp/session.go
72.80% <76.19%> (-4.78%)
:arrow_down:

hrp/step_testcase.go
65.33% <100.00%> (-1.34%)
:arrow_down:

hrp/step_thinktime.go
69.56% <100.00%> (+1.48%)
:arrow_up:

hrp/boomer.go
78.46% <0.00%> (-3.85%)
:arrow_down:

httprunner/__init__.py

httprunner/cli.py

... and 28 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8aabf5a...eac548f. Read the comment docs.


dependency mismatch: org.slf4j

We have multiple versions the classpath/overall external-libraries

org.slf4j

slf4j-api

2.0.6
2.0.7
2.0.9

slf4j-simple

2.0.9

we should analyse where these come from and use exclusions (or explicit declaration) to force common latest version.

As we do have logback on the classpath, we should try to get rid of the slf4j-simple dependecy and configure logback.
Also: We should not export the logging backend lib, just the api

fixed with #346


Undocumented clock speeds of the MCP4728

Just a Comment and Thanks.
The library works brilliantly and there is a little trick you can make note of somewhere to save someone else the 4 hours it took for me to figure it out.  I'm sure that this would be obvious by inspection to even the most casual observer, but not me.
Reason:  I needed to generate 60Hz sine waves and by default, this isn't satis-factually possible. Iterating through a 256-point array at 100kHz clock yields a 4.2Hz waveform.  By experimentation, I learned that this is limited by the clock rate of the i2c bus.
With an ATmega328P (Nano clone), the highest clock rate supported by the i2c library (without assembly I assume) and the MCP4728 is 888kHz. The MCP itself supports a High-Speed mode of 3.4MHz but it is not obvious to me how to initialize the chip & processor & library to run that fast.
Wire.begin(100000L); sets 100kHz clock (default), 400000L sets 400kHz and although undocumented anywhere I discovered that  800000L sets the clock to 800kHz and it works.
But we can squeeze a bit more out of it. twi.c (/Users/jm/Library/Arduino15/packages/arduino/hardware/avr/1.6.23/libraries/Wire/src/utility/twi.c) contains the following code:
  // initialize twi prescaler and bit rate
  cbi(TWSR, TWPS0);
  cbi(TWSR, TWPS1);
  TWBR = ((F_CPU / TWI_FREQ) - 16) / 2;

  /* twi bit rate formula from atmega128 manual pg 204
  SCL Frequency = CPU Clock Frequency / (16 + (2 * TWBR))
  note: TWBR should be 10 or higher for master mode
  It is 72 for a 16mhz Wiring board with 100kHz TWI */

We know that this calculation can be overridden by specifying TWBR explicitly after
Wire.begin().  A little pencil and paper work tells us that 100kHz clock is TWBR = 72, 400kHz clock is TWBR=12 and TWBR = 2 yields 800kHz.  Since TWBR can't be negative, setting it to 1 gives us a clock of 888kHz.  With this I can generate three passable albeit ugly, 60Hz sine waves, 120deg offset (power system simulation).

FYI: The library also works (unmodified) with the Adafruit M0 Trinket (Atmel ATSAMD21).  TWBR is not recognized and i2c bus clock speeds of up to 1.4MHz are possible.
e.g.: Wire.setClock(1400000L);

I'm so sorry for late reply. Thank you for your information! I also discovered we can set clock by setClock(), thanks to your advice. Thank you so much again!


Using Docket.enableUrlTemplating(true) causes unnecessary curly braces in request URL.

I am using springfox-swagger-ui-2.6.0.jar
On using following configuration with  .enableUrlTemplating(true), the request URL has curly braces appended and such a URL is incorrect and cannot be processed.
return new Docket(DocumentationType.SWAGGER_2)
.select()
.apis(RequestHandlerSelectors.any())
.paths(PathSelectors.any())
.build()
.enableUrlTemplating(true)
.globalOperationParameters(parameters);
For example the REQUEST URL formed in swagger-ui.html will look like
http://jkaur2k12:61400/rest/service/admin/attributes/2{?loginname,loginpassword,view}& loginname=Admin&loginpassword=Admin&view=xmlView

@JaspreetKaur16 Your choices are to either list all the operations or Have the docket support url templating. Cant have both.

Now if we look at the attached image of issue, I believe in following Request URL the part
{?loginname,loginpassword,view} is unneccessary. Ideally the Request URL should be this
http://jkaur2k12:61400/rest/service/admin/attributes/2?loginname=Admin&loginpassword=Admin&view=xmlView
rather than the following incorrcet one.
http://jkaur2k12:61400/rest/service/admin/attributes/2{?loginname,loginpassword,view}& loginname=Admin&loginpassword=Admin&view=xmlView
How to fix this ?
Thanks,
Jaspreet


I have the following structure:
Main
   `-- Panel
           |-- List
           `-- Input

Main component is wrapped with HotKeys and the mixin applied. Currently I'm playing with one key - f.
Main has only keyMap defined, but the handler for f is defined in Input component. The handler gets called only if I place focus on Input.
The idea is to be able to call handler of the hotkey on Input, if the focus is placed on its parent, in this case the parent is Main.
The idea is to be able to place focus on the input from any place of the app.
Is this possible, did I miss something, or I should pick another way?

I think your issue is similar to my own and I believe it's caused by the focus is on the body when the app initially renders. It's not possible to wrap the body in a HotKeys component for a global map and I haven't yet found a way to have focus start lower down in the tree on app load so that my handlers will notice events. I didn't really understand how the mixin played into all this so that may be the answer somehow.

Hey guys I completely forgot until a colleague sent me a snippet of my own code that we allow attaching the HotKeys to an alternative DOMElement instance. I think this might address your issue @coldpour.
For example you could do <HotKeys attach={window} />. See reference here: https://github.com/Chrisui/react-hotkeys/blob/master/lib/HotKeys.js#L61
As for your issue @kuzzmi I think you are expecting the reverse propagation of what the event system actually provides. If a child (Input) is focused we can say it's parent (Main) is focused too but we cannot say the same in reverse. Therefore if you want to handle a key event that occurs when Input or Panel or Main is in focus then put the handler on Main.
Sorry for the slow response!

Closing since this was opened a while ago! Feel free to re-open if you need further help. :)

@chrisui, sorry it took so long, but I just got around to trying out your suggestion and it didn't work for me. I tried the window, body, document, document.body and all those values in strings as well. No matter what I put in the attach prop, I always had to click on the screen before the handlers would fire. I ended up having to do document.body.addEventListener('keydown', this.onKeyDown.bind(this)); inside componentDidMount to get my handler to fire without clicking first.

@chrisui : I try with your suggestion but nothing happens, I always need to focus on element
although try the same with @coldpour

If all you need is to listen after page load (without having to focus on the React HotKeys element by clicking on it), then you can just set the React HotKeys element to be your document.activeElement, like so:
componentDidMount() {
    this.mainDiv.focus();
}
render() {
    return (
        <HotKeys handlers={...}>
            <div ref={(c) => { this.mainDiv = c; }} />
        </HotKeys>
    )
}

For anyone else who comes across global keybinding focus issues, it seems that this has been comprehensively addressed in the 2.0.0 release with the GlobalHotKeys component. I switched to that for my app and all of my global keybinding issues have been solved!


test: azureAD token lifetime too short to run deployment tests

Ref: Azure/login/issues/180
Attempted to fix by assigning custom policy to SPN as documented here: https://docs.microsoft.com/en-us/azure/active-directory/develop/configure-token-lifetimes

fixed by #141


Beta calibration curve looks different from www

beta:

www:

Let's check the maths, @lsabor.

Somewhat fixed: the displayed Beta one is for the metaculus prediction, here is the recency-weighted community prediction version:

here is the updated rewrite version:

I've moved this to medium priority. @SylvainChevalier feel free to move it back to high priority.

As discussed yesterday, the top bin being bad and different is concerning.

@SylvainChevalier are the bucket the same here ? it looks like the top is different in the new one (100 vs 95)

Update:
Recency Weighted main site:

Recency Weighted rewrite:

MP main site:

MP rewrite:

Here's what happens when I set the forecast horizon to be from open to actual close time not scheduled close time:

What's going on here is that "coverage" in the main site for ComboPrediction is 100% if a forecast starts at open even if the question resolves early. Looks like that was our culprit...

Well, that's definitely not the whole difference as my personal track record is still off:

Note the 4th and 6th bins are much closer in the main site than the rewrite.

merged above changes with notes. still more to do though

@lsabor what was left to do?

I haven't looked at this since Sep 17. It's still different, and I never got to the bottom of why.
I think eventually this should be looked at seriously, and the math checked over...

connected: #1480


Show type hover

Fixes #65

Force-pushed after rebasing this branch onto main (with the new update). It resulted in some loss of data, and no longer shows the comments the @anka-213 has written previously. The comments can be found here:
https://github.com/smucclaw/baby-l4/commit/e3ea078813f842e44654447ddd8e75bebdefb7d3


Add benchmark script for CI

Does not yet use a dedicated bare metal benchmarking server
Pushes benchmark results to cloud

Codecov Report

Merging #74 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #74   +/-   ##
=========================================
  Coverage     75.77%   75.77%
  Complexity      723      723
=========================================
  Files            78       78
  Lines          2621     2621
  Branches        233      233
=========================================
  Hits           1986     1986
  Misses          497      497
  Partials        138      138

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 65f4313...5a92352. Read the comment docs.


Electron: merge Go network stack

Merges the Go network stack for Windows and Linux clients from the go-tun2socks branch, alongside the existing network stack. See #688.
Defines common interfaces for Shadowsocks and Tun2socks implementations.
TODO: switch the network stack based on a build flag; migrate Electron app build to webpack so we can use static compile-time constants.

Thank you for the feeback. I have addressed the comments in #978.


Possible memory leak in ParaViewDataCollection->Save()

Hello everyone!
I've found an overflow of memory on cluster nodes when I compute problems with large meshes. I checked a memory consumption by htop command, and I saw a step-like increase of memory usage in moments of data writing by function ParaViewDataCollection->Save().
To reproduce a problem, I added ParaViewDataCollection into Example 18; here, you can find pieces of code:
   args.AddOption(&vis_steps, "-vs", "--visualization-steps",
                  "Visualize every n-th timestep.");
   **args.AddOption(&paraview, "-paraview", "--paraview-data", "-no-paraview",
                  "--no-paraview-data",
                  "Enable or disable ParaView visu.");**

   args.Parse();

.....
  **ParaViewDataCollection *pd = NULL;
    if (paraview)
    {
        pd = new ParaViewDataCollection("PV", &pmesh);
        pd->RegisterField("mom", &mom);

        pd->SetLevelsOfDetail(1);
        pd->SetCycle(0);
        pd->SetTime(0.0);
        pd->Save();

        if (myRank == 0) cout << "Paraview OK" << endl;
    }**

   // 9. Set up the nonlinear form corresponding to the DG discretization of the
   //    flux divergence, and assemble the corresponding mass matrix.
   MixedBilinearForm Aflux(&dfes, &fes);

....
   if (done || ti % vis_steps == 0)
      {
         **if (paraview)
            {
                pd->SetCycle(ti);
                pd->SetTime(t);
                pd->Save();
                if (mpi.Root()) {cout << "ParaView OK\n";}
            }**

I've run an application by the following command line (ParaView visualisation is switched on by default):
./ex18par -m ../../data/periodic-square.mesh -o 1 -rs 5 -rp 0 -s 2 -vs 1 -tf 1 -dt 0.01 -c -1
There were 100 time steps; I've saved ParaView frame every time step; in the end, I've seen near 40 Mb extra memory usage by htop.
When I didn't write ParaView frames, memory usage was the constant value.
The problem exists both in serial and parallel mode.
Therefore, it can be two ways: some memory leak in ParaviewDataCollection->Save() function or some wrong usage of ParaviewDataCollection. What can we do?
I'll be very-very thankful for help!
--
Best regards, Victoria

Hello @vkorchagova,
I am not able to reproduce this problem. I tried profiling the code with valgrind --tool=massif, but the memory usage appeared roughly constant over time.
ParaView output will require some memory overhead, but it shouldn't increase meaningfully as you run the code for more time steps.
If you have valgrind, could you try profiling a minimal example with massif and seeing if ms_print shows an increase? This is what I am seeing:
    MB
42.85^        #
     |        #@::::@::::::::::::::::::::::@:::@:::@@:::@:::@::::::@::::::@::::::@:::
     |       :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |       :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |       :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |       :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |       :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |     @@:#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |    @@ :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
     |::@:@@ :#@::: @::: ::::::::::: :::: :@:::@:::@ : :@:::@::::::@::::::@::::::@:::
   0 +------------------------------------------------------------------------------>Gi
     0                                                                          190.7

Hello @pazner,
I'm so sorry -- the problem was in hardware on my cluster. When remedied works were carried out, Valgrind gave me the roughly constant memory usage too.
Many thanks for your patience!


iroh-net: valgrind error: invalid atomic reads

This looks like an issue in https://github.com/rust-lang/futures-rs/blob/master/futures-util/src/stream/futures_unordered/ready_to_run_queue.rs and more specifically in the way drop is implemented.
This might or might not be responsible for the malloc crashes we have been seeing in CI
==1716606== Thread 25 magic_endpoint::
==1716606== Invalid read of size 8
==1716606==    at 0x38B867: core::sync::atomic::atomic_load (atomic.rs:3185)
==1716606==    by 0x38FBA7: core::sync::atomic::AtomicPtr<T>::load (atomic.rs:1325)
==1716606==    by 0xBAB2CB: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::dequeue (ready_to_run_queue.rs:60)
==1716606==    by 0xBA9FD7: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::clear (ready_to_run_queue.rs:102)
==1716606==    by 0xE9BA39: <futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut> as core::ops::drop::Drop>::drop (ready_to_run_queue.rs:119)
==1716606==    by 0xEFD092: core::ptr::drop_in_place<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0x94989D: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1749)
==1716606==    by 0xF42203: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xF02599: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xEF970E: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0xF1DAA6: core::ptr::drop_in_place<futures_util::stream::stream::buffer_unordered::BufferUnordered<futures_util::stream::stream::map::Map<futures_util::stream::iter::Iter<alloc::vec::into_iter::IntoIter<alloc::sync::Arc<iroh_net::derp::map::DerpNode>>>,iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xF3FABE: core::ptr::drop_in_place<iroh_net::derp::http::client::Client::dial_region::{{closure}}> (client.rs:680)
==1716606==  Address 0x9eef4a8 is 1,352 bytes inside a block of size 1,368 free'd
==1716606==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0x10552AA: <alloc::alloc::Global as core::alloc::Allocator>::deallocate (alloc.rs:117)
==1716606==    by 0x105523F: <&A as core::alloc::Allocator>::deallocate (mod.rs:386)
==1716606==    by 0xF65894: <alloc::sync::Weak<T,A> as core::ops::drop::Drop>::drop (sync.rs:3001)
==1716606==    by 0xF032A9: core::ptr::drop_in_place<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>,&alloc::alloc::Global>> (mod.rs:497)
==1716606==    by 0x94A158: drop<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>, &alloc::alloc::Global>> (mod.rs:987)
==1716606==    by 0x94A158: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1755)
==1716606==    by 0xF41943: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xEFCE39: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xC6987E: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::release_task (mod.rs:283)
==1716606==    by 0xC6AAA0: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::clear_head_all (mod.rs:573)
==1716606==    by 0xE9B7F9: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as core::ops::drop::Drop>::drop (mod.rs:584)
==1716606==    by 0xEF9702: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==  Block was alloc'd at
==1716606==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0xB21C77: alloc (alloc.rs:98)
==1716606==    by 0xB21C77: alloc::alloc::Global::alloc_impl (alloc.rs:181)
==1716606==    by 0xB21A26: alloc::alloc::exchange_malloc (alloc.rs:241)
==1716606==    by 0x942BCE: new<alloc::sync::ArcInner<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>>> (boxed.rs:217)
==1716606==    by 0x942BCE: alloc::sync::Arc<T>::new (sync.rs:388)
==1716606==    by 0xC6EBB2: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::push (mod.rs:167)
==1716606==    by 0xAE5A1E: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next (buffer_unordered.rs:69)
==1716606==    by 0xAE5C58: futures_util::stream::stream::StreamExt::poll_next_unpin (mod.rs:1632)
==1716606==    by 0x4A2716: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll (next.rs:32)
==1716606==    by 0xE55E8D: iroh_net::derp::http::client::Client::dial_region::{{closure}} (client.rs:662)
==1716606==    by 0xE7380F: iroh_net::derp::http::client::Client::connect_0::{{closure}}::{{closure}} (client.rs:477)
==1716606==    by 0xC4FB2A: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll (instrument.rs:272)
==1716606==    by 0xE7120F: iroh_net::derp::http::client::Client::connect_0::{{closure}} (client.rs:463)
==1716606==
==1716606== Invalid write of size 8
==1716606==    at 0x38E3A1: core::sync::atomic::atomic_store (atomic.rs:3170)
==1716606==    by 0x3902FA: core::sync::atomic::AtomicPtr<T>::store (atomic.rs:1355)
==1716606==    by 0xBAB677: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::enqueue (ready_to_run_queue.rs:41)
==1716606==    by 0xBAB36D: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::dequeue (ready_to_run_queue.rs:73)
==1716606==    by 0xBA9FD7: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::clear (ready_to_run_queue.rs:102)
==1716606==    by 0xE9BA39: <futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut> as core::ops::drop::Drop>::drop (ready_to_run_queue.rs:119)
==1716606==    by 0xEFD092: core::ptr::drop_in_place<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0x94989D: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1749)
==1716606==    by 0xF42203: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xF02599: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xEF970E: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0xF1DAA6: core::ptr::drop_in_place<futures_util::stream::stream::buffer_unordered::BufferUnordered<futures_util::stream::stream::map::Map<futures_util::stream::iter::Iter<alloc::vec::into_iter::IntoIter<alloc::sync::Arc<iroh_net::derp::map::DerpNode>>>,iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==  Address 0x9eef4a8 is 1,352 bytes inside a block of size 1,368 free'd
==1716606==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0x10552AA: <alloc::alloc::Global as core::alloc::Allocator>::deallocate (alloc.rs:117)
==1716606==    by 0x105523F: <&A as core::alloc::Allocator>::deallocate (mod.rs:386)
==1716606==    by 0xF65894: <alloc::sync::Weak<T,A> as core::ops::drop::Drop>::drop (sync.rs:3001)
==1716606==    by 0xF032A9: core::ptr::drop_in_place<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>,&alloc::alloc::Global>> (mod.rs:497)
==1716606==    by 0x94A158: drop<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>, &alloc::alloc::Global>> (mod.rs:987)
==1716606==    by 0x94A158: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1755)
==1716606==    by 0xF41943: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xEFCE39: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xC6987E: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::release_task (mod.rs:283)
==1716606==    by 0xC6AAA0: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::clear_head_all (mod.rs:573)
==1716606==    by 0xE9B7F9: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as core::ops::drop::Drop>::drop (mod.rs:584)
==1716606==    by 0xEF9702: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==  Block was alloc'd at
==1716606==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0xB21C77: alloc (alloc.rs:98)
==1716606==    by 0xB21C77: alloc::alloc::Global::alloc_impl (alloc.rs:181)
==1716606==    by 0xB21A26: alloc::alloc::exchange_malloc (alloc.rs:241)
==1716606==    by 0x942BCE: new<alloc::sync::ArcInner<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>>> (boxed.rs:217)
==1716606==    by 0x942BCE: alloc::sync::Arc<T>::new (sync.rs:388)
==1716606==    by 0xC6EBB2: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::push (mod.rs:167)
==1716606==    by 0xAE5A1E: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next (buffer_unordered.rs:69)
==1716606==    by 0xAE5C58: futures_util::stream::stream::StreamExt::poll_next_unpin (mod.rs:1632)
==1716606==    by 0x4A2716: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll (next.rs:32)
==1716606==    by 0xE55E8D: iroh_net::derp::http::client::Client::dial_region::{{closure}} (client.rs:662)
==1716606==    by 0xE7380F: iroh_net::derp::http::client::Client::connect_0::{{closure}}::{{closure}} (client.rs:477)
==1716606==    by 0xC4FB2A: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll (instrument.rs:272)
==1716606==    by 0xE7120F: iroh_net::derp::http::client::Client::connect_0::{{closure}} (client.rs:463)
==1716606==
==1716606== Invalid read of size 8
==1716606==    at 0x38B867: core::sync::atomic::atomic_load (atomic.rs:3185)
==1716606==    by 0x38FBA7: core::sync::atomic::AtomicPtr<T>::load (atomic.rs:1325)
==1716606==    by 0xBAB3B7: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::dequeue (ready_to_run_queue.rs:75)
==1716606==    by 0xBA9FD7: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::clear (ready_to_run_queue.rs:102)
==1716606==    by 0xE9BA39: <futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut> as core::ops::drop::Drop>::drop (ready_to_run_queue.rs:119)
==1716606==    by 0xEFD092: core::ptr::drop_in_place<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0x94989D: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1749)
==1716606==    by 0xF42203: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xF02599: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xEF970E: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0xF1DAA6: core::ptr::drop_in_place<futures_util::stream::stream::buffer_unordered::BufferUnordered<futures_util::stream::stream::map::Map<futures_util::stream::iter::Iter<alloc::vec::into_iter::IntoIter<alloc::sync::Arc<iroh_net::derp::map::DerpNode>>>,iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xF3FABE: core::ptr::drop_in_place<iroh_net::derp::http::client::Client::dial_region::{{closure}}> (client.rs:680)
==1716606==  Address 0x9eef4a8 is 1,352 bytes inside a block of size 1,368 free'd
==1716606==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0x10552AA: <alloc::alloc::Global as core::alloc::Allocator>::deallocate (alloc.rs:117)
==1716606==    by 0x105523F: <&A as core::alloc::Allocator>::deallocate (mod.rs:386)
==1716606==    by 0xF65894: <alloc::sync::Weak<T,A> as core::ops::drop::Drop>::drop (sync.rs:3001)
==1716606==    by 0xF032A9: core::ptr::drop_in_place<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>,&alloc::alloc::Global>> (mod.rs:497)
==1716606==    by 0x94A158: drop<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>, &alloc::alloc::Global>> (mod.rs:987)
==1716606==    by 0x94A158: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1755)
==1716606==    by 0xF41943: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xEFCE39: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xC6987E: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::release_task (mod.rs:283)
==1716606==    by 0xC6AAA0: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::clear_head_all (mod.rs:573)
==1716606==    by 0xE9B7F9: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as core::ops::drop::Drop>::drop (mod.rs:584)
==1716606==    by 0xEF9702: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==  Block was alloc'd at
==1716606==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0xB21C77: alloc (alloc.rs:98)
==1716606==    by 0xB21C77: alloc::alloc::Global::alloc_impl (alloc.rs:181)
==1716606==    by 0xB21A26: alloc::alloc::exchange_malloc (alloc.rs:241)
==1716606==    by 0x942BCE: new<alloc::sync::ArcInner<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>>> (boxed.rs:217)
==1716606==    by 0x942BCE: alloc::sync::Arc<T>::new (sync.rs:388)
==1716606==    by 0xC6EBB2: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::push (mod.rs:167)
==1716606==    by 0xAE5A1E: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next (buffer_unordered.rs:69)
==1716606==    by 0xAE5C58: futures_util::stream::stream::StreamExt::poll_next_unpin (mod.rs:1632)
==1716606==    by 0x4A2716: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll (next.rs:32)
==1716606==    by 0xE55E8D: iroh_net::derp::http::client::Client::dial_region::{{closure}} (client.rs:662)
==1716606==    by 0xE7380F: iroh_net::derp::http::client::Client::connect_0::{{closure}}::{{closure}} (client.rs:477)
==1716606==    by 0xC4FB2A: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll (instrument.rs:272)
==1716606==    by 0xE7120F: iroh_net::derp::http::client::Client::connect_0::{{closure}} (client.rs:463)
==1716606==
==1716606== Invalid read of size 8
==1716606==    at 0xF41928: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (atomic.rs:3236)
==1716606==    by 0xEFCE39: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xD6A51C: core::mem::drop (mod.rs:987)
==1716606==    by 0xBAA01A: futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut>::clear (ready_to_run_queue.rs:105)
==1716606==    by 0xE9BA39: <futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<Fut> as core::ops::drop::Drop>::drop (ready_to_run_queue.rs:119)
==1716606==    by 0xEFD092: core::ptr::drop_in_place<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0x94989D: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1749)
==1716606==    by 0xF42203: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xF02599: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::ready_to_run_queue::ReadyToRunQueue<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xEF970E: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==    by 0xF1DAA6: core::ptr::drop_in_place<futures_util::stream::stream::buffer_unordered::BufferUnordered<futures_util::stream::stream::map::Map<futures_util::stream::iter::Iter<alloc::vec::into_iter::IntoIter<alloc::sync::Arc<iroh_net::derp::map::DerpNode>>>,iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xF3FABE: core::ptr::drop_in_place<iroh_net::derp::http::client::Client::dial_region::{{closure}}> (client.rs:680)
==1716606==  Address 0x9eeef60 is 0 bytes inside a block of size 1,368 free'd
==1716606==    at 0x484B27F: free (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0x10552AA: <alloc::alloc::Global as core::alloc::Allocator>::deallocate (alloc.rs:117)
==1716606==    by 0x105523F: <&A as core::alloc::Allocator>::deallocate (mod.rs:386)
==1716606==    by 0xF65894: <alloc::sync::Weak<T,A> as core::ops::drop::Drop>::drop (sync.rs:3001)
==1716606==    by 0xF032A9: core::ptr::drop_in_place<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>,&alloc::alloc::Global>> (mod.rs:497)
==1716606==    by 0x94A158: drop<alloc::sync::Weak<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>, &alloc::alloc::Global>> (mod.rs:987)
==1716606==    by 0x94A158: alloc::sync::Arc<T,A>::drop_slow (sync.rs:1755)
==1716606==    by 0xF41943: <alloc::sync::Arc<T,A> as core::ops::drop::Drop>::drop (sync.rs:2405)
==1716606==    by 0xEFCE39: core::ptr::drop_in_place<alloc::sync::Arc<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>>> (mod.rs:497)
==1716606==    by 0xC6987E: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::release_task (mod.rs:283)
==1716606==    by 0xC6AAA0: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::clear_head_all (mod.rs:573)
==1716606==    by 0xE9B7F9: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as core::ops::drop::Drop>::drop (mod.rs:584)
==1716606==    by 0xEF9702: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<iroh_net::derp::http::client::Client::dial_region::{{closure}}::{{closure}}::{{closure}}>> (mod.rs:497)
==1716606==  Block was alloc'd at
==1716606==    at 0x4848899: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==1716606==    by 0xB21C77: alloc (alloc.rs:98)
==1716606==    by 0xB21C77: alloc::alloc::Global::alloc_impl (alloc.rs:181)
==1716606==    by 0xB21A26: alloc::alloc::exchange_malloc (alloc.rs:241)
==1716606==    by 0x942BCE: new<alloc::sync::ArcInner<futures_util::stream::futures_unordered::task::Task<iroh_net::derp::http::client::{impl#5}::dial_region::{async_fn#0}::{closure#0}::{async_block_env#0}>>> (boxed.rs:217)
==1716606==    by 0x942BCE: alloc::sync::Arc<T>::new (sync.rs:388)
==1716606==    by 0xC6EBB2: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::push (mod.rs:167)
==1716606==    by 0xAE5A1E: <futures_util::stream::stream::buffer_unordered::BufferUnordered<St> as futures_core::stream::Stream>::poll_next (buffer_unordered.rs:69)
==1716606==    by 0xAE5C58: futures_util::stream::stream::StreamExt::poll_next_unpin (mod.rs:1632)
==1716606==    by 0x4A2716: <futures_util::stream::stream::next::Next<St> as core::future::future::Future>::poll (next.rs:32)
==1716606==    by 0xE55E8D: iroh_net::derp::http::client::Client::dial_region::{{closure}} (client.rs:662)
==1716606==    by 0xE7380F: iroh_net::derp::http::client::Client::connect_0::{{closure}}::{{closure}} (client.rs:477)
==1716606==    by 0xC4FB2A: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll (instrument.rs:272)
==1716606==    by 0xE7120F: iroh_net::derp::http::client::Client::connect_0::{{closure}} (client.rs:463)
==1716606==

Fixed in #1647


[WinError 2] The system cannot find the file specified

Please help me!

My guess is you dont have gcc/g++ available in path.


Why session not saving automatic?

Hi, Thank you for opening this issue. Regarding this, can you tell what session you are using? It you use tgsnake it will automatically saved when you kill the app (CTRL + C).

const storeSession = new Storages.StringSession('bot_session');

Other packages will prompt me to login if not exist but I got
/node_modules/@tgsnake/core/lib/src/storage/StringSession.js:71                                                   throw new Error(`Invalid String Session`);                                                                      ^
Error: Invalid String Session                            at new StringSession

Hi, a little clarification, string session is not same store session. String session is string which can be used to log in, it will look like:
BVw2QwAAF1......AcmRAA

You can use string from Telethon, Pyrogram, GramJS.
Another option you can use is RedisSession.
Regarding the error, i see you didn't enter the correct string. Instead of the string session, you fill it with the session name (bot_session). This why error happens.

Ok you only support string session?

Also this not available
await client.connect()
await client.checkAuthorization()

Ok you only support string session?

We support Redis based session and File based session. For file based session you can import from tgsnake.

Also this not available
await client.connect()
await client.checkAuthorization()

Next version I have added client.connect I will push the latest update to NPM soon after doing some testing. To check authorization, we don't have it. To check whether the user is authorized or not you can do get me if it is successful then the user is authorized.
If you have further questions, you can contact me personally or in the group: https://t.me/tgsnakechat

My problem is I don't want to be logging in everytime I run the script

If you use string session, you can export it and reuse it. You will not be asked to log in again after that.

import { Client, Raw, Storages } from '@tgsnake/core';
process.env.LOGLEVEL = 'debug'; // set log level, see @tgsnake/log for more information.
const client = new Client(
  // you can fill with Telethon or Pyrogram string session.
  new Storages.StringSession(''),
  apiHash,
  apiId,
);

(async () => {
  // if you already pass the string session, don't fill any arguments in start function, leave it empty.

  await client.start({
    botToken: '', // if you want login as bot, you can login as user too.
    /* Remove "botToken" if you want to login as user.
    phoneNumber : async () => {}, // Phone number with international phone code (include plus sign (+)) will be used to login, the return of function must be a string.
    code : async () => {}, // OTP code, the return of function must be a string.
    password : async () => {}, // if you account has 2FA, the return of function must be a string.
    authError : async (error) => {} // when error BadRequest attempt, this function will be running.
    */
  });
  // exporting session
  console.log(await client.exportSession())
})();

Then in your terminal will be show string session like:
BVw2QwAAF1......AcmRAA

You can use it to prevent re-login every time.
new Storages.StringSession('BVw2QwAAF1......AcmRAA')

async function startTelegramClient(page) {
  const storeSession = new Storages.StringSession('bot_session');

  const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  console.log("Starting Telegram client...");

  const client = new Client(
    storeSession,
    apiHash,
    apiId,
  );

  try {
    await client.start({
      phoneNumber: async () => phoneNumber,
      password: async () => password,
      code: async () =>
        new Promise((resolve) =>
          rl.question("Please enter the code you received: ", resolve)
        ),
      authError: (err) => console.log("Error during authorization:", err),
    });

    const exported = await client.exportSession();
    console.log(exported);
    console.log("Telegram client is ready and listening for new messages...");

    // Register event handler for new messages
    client.addEventHandler(
      async (event) => {
        if (event.isChannel) {
          const message = event.message.message;
        }
      },
      new NewMessage({ fromUsers: [signalChannel] })
    );

  } catch (error) {
    console.log("An error occurred:", error);
  }
}

Invalid string session, you must pass it with blank string or string generated from client.exportSession
- const storeSession = new Storages.StringSession('bot_session');
+ const storeSession = new Storages.StringSession('');

Invalid handler, we don't have client.addEventHandler but we have client.addHandler.
-     client.addEventHandler(
-      async (event) => {
-        if (event.isChannel) {
-          const message = event.message.message;
-        }
-      },
-      new NewMessage({ fromUsers: [signalChannel] })
-    );
+    client.addHandler(
+      async (update) => {
+        console.log(update)
+      }
+    );

As with the first aim, this framework was formed so that it is not binding and frees users to be creative. As the name, 'core' means only basic features are available. Basically this framework was created as the basis for tgsnake. It's better to use tgsnake if you want it simpler.

tgsnake is for telegram bot

tgsnake is for telegram bot

No, You are wrong.
Tgsnake can be used to log in as a user or bot.
Tgsnake is not using bot api, it using MTProto.

Where to pass phone number and password

Running your client, it will automatically ask the login information.

I can't pass it directly?

@Jobians yes, but it is a nice idea.. I'll add it in the next update..

const client = new Snake({
    apiHash,
    apiId,
    logLevel: 'none',
    login: {
      sessionName: 'session',
      forceDotSession: true,
    },
    clientOptions: {
      phoneNumber: async () => phoneNumber,
      password: async () => password,
      code: async () =>
        new Promise((resolve) =>
          rl.question("Please enter the code you received: ", resolve)
        ),
      authError: (err) => console.log("Error during authorization:", err),
    },
  });

select the login option, bot for login as bot, user for login as user. For NewMessage error, it not a mistake on the framework side >>.

So now not possible to pass phone number and password directly?

for clientOptions you can found on: https://tgsnake.js.org/en/guide/client-options

So now not possible to pass phone number and password directly?

yes.

But it available in @tgsnake/core

But it available in @tgsnake/core

Yes, it is available in @tgsnake/core but we have implemented it in tgsnake using the input terminal. Why do you want to pass phone number and password directly? You still can login as user without pass phone number and password directly.
For now, if you really really want to pass phone number and password directly you can build a login plugin.
const { Clients, Client } = require('@tgsnake/core');

function myPlugin(api){
  api.addEventHandler('onLogin', async (app) => {
    app._client = new Client(
      app._options.login.session,
      app._options.apiHash,
      app._options.apiId,
      all._options.clientOptions
    )
    await app._options.login.session.load();
    if(!app._options.login.session?.authKey && app._options.login?.session.userId === undefined){
      const user = await app._client.start({
        phoneNumber, password, code
      })
      await app._options.login.session.save()
      return user
    } else {
      await Clients.Session.connect(app._client)
      return await Clients.Auth.getMe(app._client)
    }
  })
}

usage:
const client = new Snake({
    apiHash,
    apiId,
    logLevel: 'none',
    login: {
      sessionName: 'session',
      forceDotSession: true,
    },
    clientOptions: {},
    plugins: [myPlugin]
  });

Ok thanks brother ♥

Your welcome, If you need further help, you can open an issue or ask in our Telegram group.
Have a nice day!

Hope you implement it in future 😁🤝

Please how to process only my channel update "@mychacnnel"
client.addHandler((update) => {
      let fupdate = update.updates;
      console.log('Update is an array:', fupdate);
console.log('Length of update array:', fupdate.length);

if (fupdate.length > 0) {
  console.log('First item in update array:', fupdate[0]);
} else {
  console.log('The update array is empty.');
}
}

Update is an array: [                                  {                                                      _: 'UpdateNewChannelMessage',                        message: {                                             _: 'Message',                                        out: false,                                          mentioned: false,
mediaUnread: false,                                  silent: false,                                       post: true,                                          fromScheduled: false,                                legacy: false,                                       editHide: false,                                     pinned: false,                                       noforwards: false,                                   invertMedia: false,                                  offline: false,                                      id: 29662,
peerId: [Object],                                    date:<PHONE_NUMBER>,                                    message: 'Helleo channel',                      entities: [],
views: 1,                                            forwards: 0,
restrictionReason: []                              },
pts: 31316,                                          ptsCount: 1
}                                                  ]
Length of update array: 1                            First item in update array: {                          _: 'UpdateNewChannelMessage',                        message: {                                             _: 'Message',                                        out: false,                                          mentioned: false,                                    mediaUnread: false,                                  silent: false,                                       post: true,                                          fromScheduled: false,                                legacy: false,                                       editHide: false,                                     pinned: false,                                       noforwards: false,                                   invertMedia: false,                                  offline: false,                                      id: 29662,                                           peerId: { _: 'PeerChannel', channelId:<PHONE_NUMBER>n },                                                     date:<PHONE_NUMBER>,                                    message: 'G Message ok',                      entities: [],                                        views: 1,                                            forwards: 0,                                         restrictionReason: []                              },                                                   pts: 31316,                                          ptsCount: 1                                        }                                                    Update is an array: undefined                        /root/betbot/src/telegram/app.js:52                  console.log('Length of update array:', fupdate.length);                                                                                ^                                                          TypeError: Cannot read properties of undefined (reading 'length')

I finally code it
client.addHandler((update) => {
      if (update instanceof Raw.Updates || update instanceof Raw.UpdatesCombined) {
        const { updates, chats, users } = update;
        for (const _update of updates) {
          if (_update instanceof Raw.UpdateNewChannelMessage){
            for (const _chat of chats) {
              if (_chat.username === signalChannel) {
                const _message = _update.message.message;
                console.log(_message);
              }
            }
          }
        }
      }
    });


why ldd  libtensorflow_cc.so can not find library related to tensorrt ?

I have build tensorflow c++ library from source code of tensorflow1.7-release vision, but when i ldd this so, i can not find any information about tenosrrt, why? And when i configure, i have designated the TRT library's path, and i use bazel to build ,the command is like this:
bazel build --config=cuda --config=monolithic //tensorflow:libtensorflow_cc.so

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Hi @sgzqc,
Right now tensorrt module behaves like a plugin and will become available to the system when you import the python module. In order to not to put it as a requirement on to the users who want to do pip install tensorflow-gpu. If you want to use TF-TRT in c++ you need to link your application to libraries produced by tensorrt package.
Sami

@sgzqc,
As I mentioned above, tensorflow-trt integration is designed as plugin and not linked to tensorflow library, even if you do monolithic builds.
Sami

@samikama  Thanks! If i want to use TF-TRT to do c++ inference, i should link both  two librarys as you say above?

Yes, you need to link against the libraries in the contrib/tensorrt as well. converters for the conversion and kernels for the ops.

I am closing this issue, since it's not a bug or feature request -- if you have additional installation or build issues, please ask a question on Stack Overflow, which has more features to help everyone share problems and solutions. Thanks!
@samikama, thanks for helping out!


Default selected when it shouldn't be

What version of CUE are you using (cue version)?
v0.4.2 and v0.4.3-beta.1
What did you do?
Note the score field in reduced, 0 is always selected despite a valid value in _data
exec cue eval reduce.cue data.cue

-- reduce.cue --
package data

reduced: {
  for key,val in _data {
    (key): {
      vulns: [ for i,vuln in val {
        severity: vuln.vulnerability.effectiveSeverity
        score: vuln.vulnerability.cvssScore | *0
        short: vuln.vulnerability.shortDescription
        urls: [ for _,el in vuln.vulnerability.relatedUrls { el.url }]
      }]
    }
  }
}

-- data.cue --
package data

_data: "data/2022-03-14/alpine-3.7": [
  {
    "createTime": "2022-03-14T17:27:30.862513Z",
    "kind": "VULNERABILITY",
    "name": "projects/ferrum-dev/locations/us/occurrences/23098fa6-7bfa-42b6-8c99-4fb7449285e8",
    "noteName": "projects/goog-vulnz/notes/CVE-2019-14697",
    "resourceUri": "us.gcr.io/ferrum-dev/alpine:3.7",
    "updateTime": "2022-03-14T17:27:30.862513Z",
    "vulnerability": {
      "cvssScore": 7.5,
      "cvssv3": {
        "attackComplexity": "ATTACK_COMPLEXITY_LOW",
        "attackVector": "ATTACK_VECTOR_NETWORK",
        "availabilityImpact": "IMPACT_HIGH",
        "baseScore": 9.8,
        "confidentialityImpact": "IMPACT_HIGH",
        "exploitabilityScore": 3.9,
        "impactScore": 5.9,
        "integrityImpact": "IMPACT_HIGH",
        "privilegesRequired": "PRIVILEGES_REQUIRED_NONE",
        "scope": "SCOPE_UNCHANGED",
        "userInteraction": "USER_INTERACTION_NONE"
      },
      "effectiveSeverity": "HIGH",
      "packageIssue": [
        {
          "affectedCpeUri": "cpe:/o:alpine:alpine_linux:3.7",
          "affectedPackage": "musl",
          "affectedVersion": {
            "fullName": "1.1.18-r3",
            "kind": "NORMAL",
            "name": "1.1.18",
            "revision": "r3"
          },
          "effectiveSeverity": "HIGH",
          "fixAvailable": true,
          "fixedCpeUri": "cpe:/o:alpine:alpine_linux:3.7",
          "fixedPackage": "musl",
          "fixedVersion": {
            "fullName": "1.1.18-r4",
            "kind": "NORMAL",
            "name": "1.1.18",
            "revision": "r4"
          }
        }
      ],
      "relatedUrls": [
        {
          "label": "More Info",
          "url": "https://security-tracker.debian.org/tracker/CVE-2019-14697"
        },
        {
          "label": "More Info",
          "url": "http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-14697"
        },
        {
          "label": "More Info",
          "url": "https://nvd.nist.gov/vuln/detail/CVE-2019-14697"
        }
      ],
      "severity": "HIGH",
      "shortDescription": "CVE-2019-14697"
    }
  }
]

What did you expect to see?
reduced: {
    "data/2022-03-14/alpine-3.7": {
        vulns: [{
            severity: "HIGH"
            score:    7.5
            short:    "CVE-2019-14697"
            urls: ["https://security-tracker.debian.org/tracker/CVE-2019-14697", "http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-14697", "https://nvd.nist.gov/vuln/detail/CVE-2019-14697"]
        }]
    }
}

What did you see instead?
reduced: {
    "data/2022-03-14/alpine-3.7": {
        vulns: [{
            severity: "HIGH"
            score:    0
            short:    "CVE-2019-14697"
            urls: ["https://security-tracker.debian.org/tracker/CVE-2019-14697", "http://people.ubuntu.com/~ubuntu-security/cve/CVE-2019-14697", "https://nvd.nist.gov/vuln/detail/CVE-2019-14697"]
        }]
    }
}

Here's a reduced example that illustrates the issue:
exec cue export c.cue
cmp stdout expect-stdout

-- expect-stdout --
{
    "a": 1,
    "b": 1
}
-- c.cue --
a: b | *0
b: 1

@verdverm

Note the score field in reduced, 0 is always selected despite a valid value in _data

AFAICS this is all working as intended. From the spec:
Expression               Value-default pair     Rules applied
*"tcp" | "udp"           ⟨"tcp"|"udp", "tcp"⟩    M1, D1

This corresponds to your example *0 | 7.5, so the concrete result is 0 not 7.5.
I think you'll need to use something like this :
        if vuln.vulnerability.cvssScore != _|_ {
                score: vuln.vulnerability.cvssScore
        }
        score: _ | *0


test: run e2e tests with driver v1.0.0-rc.1

Signed-off-by: Anish Ramasekar<EMAIL_ADDRESS>
Reason for Change:

Run e2e tests with driver v1.0.0-rc.1

Requirements

[x] squashed commits
[x] included documentation
[x] added unit tests and e2e tests (if applicable).

Issue Fixed:

Does this change contain code from or inspired by another project?

[ ] Yes
[ ] No

If "Yes," did you notify that project's maintainers and provide attribution?
Special Notes for Reviewers:

Codecov Report

Merging #674 (2b53c58) into master (875baee) will not change coverage.
The diff coverage is n/a.

:exclamation: Current head 2b53c58 differs from pull request most recent head c206f0e.


SAVE AS PNG

Please consider adding the SAVE AS for PNG option.  The current Photoshop Export As or the option to Save for Web does not transmit the PPI into the PNG  only Save As transmit PPI to the PNG file.   So, if you are working in an original 300PPI file and use this script to save out PNG files, they have the Adobe default 72PPI not 300DPI.  Appreciate someone reviewing this issue and enhancing the script to generate PNGs with the PPI info.

Unfortunately using the Save As functionality is exponentially slower than using the save for web option, so I don't know how much sense it makes it add it to this script, which is aimed at being fast.