Fix missing test coverage found by cargo-mutants

I heard about cargo-mutants a RustConf 2024, and thought I'd give it a go. It ran for about 2h and found the following. Maybe this is nonsense, but worth a look. It would run faster if I could convince it to only run the lib tests and not the integration tests.

[ ] taskchampion/src/server/cloud/gcp.rs:131:23: replace > with == in ::compare_and_swap in 5.5s build + 24.1s test
[ ] taskchampion/src/taskdb/snapshot.rs:31:9: replace ::expecting -> fmt::Result with Ok(Default::default()) in 5.6s build + 24.5s test
[x] taskchampion/src/storage/mod.rs:137:23: replace && with || in StorageTxn::is_empty in 5.2s build + 24.5s test
[ ] taskchampion/src/server/cloud/gcp.rs:211:36: replace < with == in <impl Iterator for ObjectIterator<'a>>::next in 5.2s build + 24.8s test
[ ] taskchampion/src/replica.rs:420:9: replace Replica::add_undo_point -> Result<()> with Ok(()) in 5.6s build + 32.2s test
[ ] taskchampion/src/task/task.rs:244:9: replace Task::get_legacy_udas -> impl Iterator<Item =(&str, &str)>+'_ with ::std::iter::once(("xyzzy", "xyzzy")) in 5.4s build + 25.1s test
[ ] taskchampion/src/task/task.rs:252:9: replace Task::get_modified -> Option with Some(Default::default()) in 5.3s build + 24.9s test
[ ] taskchampion/src/server/sync/mod.rs:168:9: replace ::add_snapshot -> Result<()> with Ok(()) in 5.3s build + 29.0s test
[ ] taskchampion/src/taskdb/sync.rs:121:46: replace == with != in sync in 5.2s build + 24.1s test
[ ] taskchampion/src/server/cloud/gcp.rs:21:70: replace == with != in is_http_error in 5.2s build + 24.6s test
[ ] taskchampion/src/task/tag.rs:28:9: replace Tag::is_synthetic -> bool with false in 5.3s build + 24.3s test
[ ] taskchampion/src/task/task.rs:244:9: replace Task::get_legacy_udas -> impl Iterator<Item =(&str, &str)>+'_ with ::std::iter::once(("xyzzy", "")) in 8.7s build + 29.2s test
[x] taskchampion/src/storage/mod.rs:135:23: replace && with || in StorageTxn::is_empty in 4.7s build + 24.5s test
[ ] taskchampion/src/server/sync/mod.rs:85:36: replace == with != in sealed_from_resp in 5.1s build + 24.3s test
[ ] taskchampion/src/server/cloud/gcp.rs:88:12: delete ! in ::del in 4.5s build + 24.2s test
[x] taskchampion/src/storage/mod.rs:133:9: replace StorageTxn::is_empty -> Result with Ok(true) in 5.5s build + 24.1s test
[ ] taskchampion/src/server/encryption.rs:164:44: replace + with * in Envelope<'a>::to_bytes in 5.2s build + 24.7s test
[ ] taskchampion/src/server/cloud/server.rs:194:14: replace < with == in CloudServer::snapshot_urgency in 5.9s build + 24.6s test
[ ] taskchampion/src/server/cloud/gcp.rs:63:9: replace ::get -> Result<Option<Vec>> with Ok(Some(vec![0])) in 4.8s build + 24.2s test
[ ] taskchampion/src/server/cloud/gcp.rs:142:27: replace != with == in ::compare_and_swap in 5.1s build + 33.6s test
[ ] taskchampion/src/server/cloud/gcp.rs:23:72: replace == with != in is_http_error in 5.8s build + 25.1s test
[ ] taskchampion/src/server/encryption.rs:145:27: replace + with * in Envelope<'a>::from_bytes in 4.5s build + 24.0s test
[ ] taskchampion/src/taskdb/mod.rs:157:9: replace TaskDb::commit_reversed_operations -> Result with Ok(false) in 5.8s build + 24.9s test
[ ] taskchampion/src/utils.rs:37:9: replace ::as_ref -> &[u8] with Vec::leak(Vec::new()) in 5.3s build + 24.4s test
[ ] taskchampion/src/server/cloud/gcp.rs:110:9: replace ::compare_and_swap -> Result with Ok(false) in 6.3s build + 24.6s test
[ ] taskchampion/src/task/tag.rs:33:9: replace Tag::is_user -> bool with false in 5.4s build + 24.1s test
[ ] taskchampion/src/task/task.rs:40:9: replace ::eq -> bool with true in 5.8s build + 24.0s test
[ ] taskchampion/src/server/cloud/gcp.rs:19:5: replace is_http_error -> bool with false in 6.4s build + 24.3s test
[x] taskchampion/src/replica.rs:106:9: replace Replica::all_task_uuids -> Result<Vec> with Ok(vec![]) in 5.1s build + 24.5s test
[ ] taskchampion/src/task/task.rs:321:9: replace Task::set_wait -> Result<()> with Ok(()) in 5.2s build + 24.3s test
[ ] taskchampion/src/server/cloud/gcp.rs:214:37: replace += with *= in <impl Iterator for ObjectIterator<'a>>::next in 5.5s build + 24.5s test
[ ] taskchampion/src/task/task.rs:244:9: replace Task::get_legacy_udas -> impl Iterator<Item =(&str, &str)>+'_ with ::std::iter::once(("", "xyzzy")) in 5.4s build + 24.8s test
[x] taskchampion/src/replica.rs:106:9: replace Replica::all_task_uuids -> Result<Vec> with Ok(vec![Default::default()]) in 5.3s build + 24.7s test
[ ] taskchampion/src/server/cloud/gcp.rs:131:23: replace > with < in ::compare_and_swap in 5.5s build + 24.8s test
[ ] taskchampion/src/server/cloud/gcp.rs:63:9: replace ::get -> Result<Option<Vec>> with Ok(None) in 4.7s build + 24.4s test
[ ] taskchampion/src/server/cloud/gcp.rs:110:9: replace ::compare_and_swap -> Result with Ok(true) in 5.4s build + 24.7s test
[ ] taskchampion/src/task/task.rs:40:9: replace ::eq -> bool with false in 5.5s build + 24.2s test
[ ] taskchampion/src/server/cloud/gcp.rs:204:9: replace <impl Iterator for ObjectIterator<'a>>::next -> OptionSelf::Item with None in 6.5s build + 24.4s test
[x] taskchampion/src/replica.rs:446:9: replace Replica::num_undo_points -> Result with Ok(1) in 5.1s build + 24.8s test
[ ] taskchampion/src/utils.rs:37:9: replace ::as_ref -> &[u8] with Vec::leak(vec![0]) in 5.2s build + 24.6s test
[ ] taskchampion/src/task/task.rs:244:9: replace Task::get_legacy_udas -> impl Iterator<Item =(&str, &str)>+'_ with ::std::iter::once(("", "")) in 6.2s build + 24.5s test
[x] taskchampion/src/replica.rs:355:12: delete ! in Replica::commit_reversed_operations in 5.3s build + 24.5s test
[ ] taskchampion/src/server/cloud/server.rs:205:9: replace CloudServer::maybe_cleanup -> Result<()> with Ok(()) in 6.0s build + 24.4s test

Looking through the rest of these, none are especially interesting or worth the time to investigate.


[Bloc] Compositing rendering should support BlElement>>#opacity:

There is no opacity layer yet

It works now


Move constructor/ move assignment operator

Can you add move constructor / move assignment operator so that EWAHBoolArray works better inside std::vector, etc.?
Please let me know if you want me to create a PR with the change.

PR invited!

Presumably, this has been resolved.

Que me eh perdido, ¿el que esta resuelto?

https://github.com/lemire/EWAHBoolArray/pull/29


Add support for map updates in laser localization filters

Related to #85.

Summary

This patch exposes a map update API for beluga::LaserLocalizationInterface2d compatible filters, and makes use of it in beluga_amcl to support map updates on an existing particle filter. Built on top of #188.
Checklist

[x] Read the contributing guidelines.
[x] Configured pre-commit and ran colcon test locally.
[x] Signed all commits for DCO.
[x] Added tests (regression tests for bugs, coverage of new code for features).
[x] Updated documentation (as needed).
[ ] Checked that CI is passing.

Rebased to solve merge conflicts.


Supporting Rails 5

Dear Refinery community 😄 ,
I've been busy with updating this gem, so the authentication will also work with Rails 5. This PR is currently using the feature/rails-5 branch for the dependencies refinerycms and refinerycms-i18n.
I've replaced except! with extract!, because except! will be deprecated in Rails 5.1. This function can be replaced with extract!, because we're only removing the :password and the password_confirmation in exclude_password_assignment_when_blank! from the user params.
The feature spec https://github.com/refinery/refinerycms-authentication-devise/blob/master/spec/features/refinery/authentication/devise/sessions_spec.rb#L119 will probably fail. This is caused by the new RefineryCMS UI, which is only less than half implemented. Maybe it's an idea that we'll remove this UI, because it's been 1,5 year and the UI isn't still fully implemented in Refinery? I'll create a special PR for this in the RefineryCMS gem, so you guys may make a call about it.

Thanks for your contribution @sjoulbak!
I agree with you to revert new admin UI at this time in order to be able to easily ship a new version of Refinery and extensions with Rails 5 support.
After the Rails 5 support we could rework on the new admin UI.
Thoughts @parndt and @stefanspicer ?

I've merged your other PR for the UI 😄

Thank you Philip for approving the PR 😄
I think that we should wait merging this PR, because we're still using the feature/rails-5 branch here in the Gemfile.

👍 thanks for that @sjoulbak

@parndt Do you agree with: https://github.com/sjoulbak/refinerycms-authentication-devise/pull/1 ?

@sjoulbak sure, but you'll need to open the PR on this repo for me to merge it 😄

Haha I only wanted your approvement 😉 It's merged into this branch

Closed in favor of #34


Releases for this repo can be of the form vX.Y.Z instead of X.Y.Z

Since you always release using X.Y.Z, go mod doesn't identify it as a valid release tag and versions your repo using the latest commit message. If you just include the v at the beginning of tags, then go mod will understand your tagging correctly.
This is how go mod identifies your tag as of now :
github.com/go-vgo/robotgo v0.0.0-20190311142943-193b4935d4b1
if you included the v in the tag, it will be like github.com/go-vgo/robotgo v0.80.0.

This will be updated later, and there is currently no bug fix version, please use the latest code.

Closed because of will be released.


BIC code format check is incorrect

BIC code can be either 8 characters long or 11 characters long
SepaQr\Data->setBic($bic) allows BIC code to be from 8 to 11 characters long (9 and 10 character long codes are invalid).
ref: https://www.swift.com/standards/data-standards/bic-business-identifier-code

PR #12 https://github.com/smhg/sepa-qr-data-php/pull/12 addresses this issue

Good catch, thanks!


Remove extra fields from UserPermissionPresenter

We added some extra fields to this presenter to get a single test (spec/requests/carto/api/visualizations_controller_spec.rb:323) to pass:
https://github.com/CartoDB/cartodb/pull/12792/commits/9e2776a46cb5beb097a69863aa54e15eed77b0d5
That test does some comparisons between different presenters that are probably not right. We could remove the fields from the UserPermissionPresenter and fix the test instead. This increases the size of responses to the visualization controller quite a bit (~10%).

Probably not worth working on it since we are considering a more in-depth revamp of this controller.


Use assets:clobber to remove assets before precompile.

We're not concerned with doing rolling deploys so let's just blow away the public/assets directory before precompile.
From: http://edgeguides.rubyonrails.org/command_line.html
"You can precompile the assets in app/assets using rake assets:precompile, and remove older compiled assets using rake assets:clean. The assets:clean task allows for rolling deploys that may still be linking to an old asset while the new assets are being built.
If you want to clear public/assets completely, you can use rake assets:clobber."

@simaishi This should fix public/assets not getting deleted between runs.
@matthewd @Fryguy is this the right thing to do?  We don't want to keep old assets around if any exist.  I remember asset:clean deleting the public/assets before, was this a rails 3 -> 4 change?

Clean used to literally do rm -rf public/assets.  I think that changed in Rails 4 to clobber and now clean just does the individual removes


Add support for the glob-style pattern matching in TypeScript 2.0

See https://github.com/Microsoft/TypeScript/pull/5980 and https://github.com/Microsoft/TypeScript/pull/8841

Now that TypeScript 2.0 beta version is released, maybe it could be possible to release beta version of gulp-typescript?

@szimek You can already use the beta of TypeScript (see readme), only globs are not supported yet.

This should be working in gulp-typescript 2.14 and 3.0

Awesome! Thanks. =)


changing validation options to support version node 12

Fixes #
changing validation options to support version node 12

Proposed Changes

refactoring source to node 12

Readiness Checklist
Author/Contributor

[x] If documentation is needed for this change, has that been included in this pull request
[x] Remember to check if code coverage decrease, if so, please implement the necessary tests

Reviewing Maintainer

[ ] Label as breaking if this is a large fundamental change
[ ] Label as either automation, bug, documentation, enhancement, infrastructure, or performance

:tada: This PR is included in version 1.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Fixed Typo

Doc said API registrations instead of APP registrations

Verified in the Azure portal; merging. @kunalbabre Thanks for submitting this fix.


Not working well with HtmlWebpackPlugin - wrong output

Current behaviour 💣
The module generated the wrong output file
Expected behaviour ☀
the index I provided in the template  config field along with the webpack output
Reproduction Example 👾
Create react app + craco
craco.config.js
const HtmlWebpackPlugin = require('html-webpack-plugin');

module.exports = () => {
    return {
        webpack: {
            plugins: [
                new HtmlWebpackPlugin({
                    hash: false,
                    template: './index.php',
                    filename: 'index.php',
                }),
            ],
        },
    };
};

The output is (index.php):
`<head><link href="/components/static/css/main.6dea0f05.chunk.css" rel="stylesheet"></head>/components/static/media/index.e8e919fb.php<script src="/components/static/js/runtime-main.3c932baa.js"></script><script src="/components/static/js/2.521c8692.chunk.js"></script><script src="/components/static/js/main.856a5df0.chunk.js"></script>`

my index.php is a lot different... it has 5k lines of code, and it has a strange line in it:
`/components/static/media/index.e8e919fb.php`

Environment 🖥
+--<EMAIL_ADDRESS>--<EMAIL_ADDRESS> --<EMAIL_ADDRESS>--<EMAIL_ADDRESS> --<EMAIL_ADDRESS>

You may have to remove the existing plugin and add it again and use index.html as filename setting:

You can easily check the current plugin configurations:

Working setup on my side:


Can't use the word "type" in a sql query

Version
1.14.0
What happened?
Can't use the word "type" in a query. Using any other column from the schema works, but outputs an error when inputting type.
Playground: https://play.sqlc.dev/p/4525691850632615a5552811c41540609e389e4596646a4744bb6466dedf81ea
Relevant log output
// Code generated by sqlc. DO NOT EDIT.
// versions:
//   sqlc v1.14.0

package db

import (

        "context"
        "database/sql"

)

    type Querier interface {
            GetTag(ctx context.Context, id int32) (Tag, error)
            GetTagsByType(ctx context.Context, type sql.NullString) ([]Tag, error)
    }

    var _ Querier = (*Queries)(nil)

# package db
error generating code: source error: 21:48: expected ), found 'type'
make: *** [sqlc] Error 1

Database schema
CREATE TABLE "public"."tags" (
    "id" int4 NOT NULL DEFAULT nextval('tags_id_seq'::regclass),
    "name" jsonb NOT NULL,
    "slug" jsonb NOT NULL,
    "type" varchar(191),
    "order_column" int4,
    "created_at" timestamp(0),
    "updated_at" timestamp(0),
    "image_url" varchar(191),
    PRIMARY KEY ("id")
);

SQL queries
-- name: GetTagsByType :many
SELECT * FROM tags
WHERE type = $1
ORDER BY name DESC;

Configuration
version: "1"
project:
    id: ""
packages:
  - name: "db"
    path: "./db/sqlc"
    queries: "./db/query/"
    schema: "./db/employbl.sql"
    engine: "postgresql"

Playground URL
https://play.sqlc.dev/p/4525691850632615a5552811c41540609e389e4596646a4744bb6466dedf81ea
What operating system are you using?
macOS
What database engines are you using?
PostgreSQL
What type of code are you generating?
Go

You need double quotes around the word type since it’s reserved
On Sat, Jul 16, 2022 at 18:47 Channing Defoe @.***>
wrote:

Version
1.14.0
What happened?
Can't use the word "type" in a query. Using any other column from the
schema works, but outputs an error when inputting type.
Playground:
https://play.sqlc.dev/p/4525691850632615a5552811c41540609e389e4596646a4744bb6466dedf81ea
Relevant log output
// Code generated by sqlc. DO NOT EDIT.
// versions:
//   sqlc v1.14.0
package db
import (
    "context"
    "database/sql"

)
type Querier interface {
        GetTag(ctx context.Context, id int32) (Tag, error)
        GetTagsByType(ctx context.Context, type sql.NullString) ([]Tag, error)
}

var _ Querier = (*Queries)(nil)

package db
error generating code: source error: 21:48: expected ), found 'type'
make: *** [sqlc] Error 1
Database schema
CREATE TABLE "public"."tags" (
"id" int4 NOT NULL DEFAULT nextval('tags_id_seq'::regclass),
"name" jsonb NOT NULL,
"slug" jsonb NOT NULL,
"type" varchar(191),
"order_column" int4,
"created_at" timestamp(0),
"updated_at" timestamp(0),
"image_url" varchar(191),
PRIMARY KEY ("id")
);
SQL queries
-- name: GetTagsByType :manySELECT * FROM tagsWHERE type = $1ORDER BY name DESC;
Configuration
version: "1"project:
id: ""packages:

name: "db"
path: "./db/sqlc"
queries: "./db/query/"
schema: "./db/employbl.sql"
engine: "postgresql"

Playground URL
https://play.sqlc.dev/p/4525691850632615a5552811c41540609e389e4596646a4744bb6466dedf81ea
What operating system are you using?
macOS
What database engines are you using?
PostgreSQL
What type of code are you generating?
Go
—
Reply to this email directly, view it on GitHub
https://github.com/kyleconroy/sqlc/issues/1749, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AML53FKEMTQUSH7WUCFCK33VUNQ3FANCNFSM53Y6K5KA
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Kevin Burke
phone:<PHONE_NUMBER> | kevin.burke.dev

@kevinburke1 Ahh of course 😅, jumped the gun on this issue, this ended up working:
-- name: GetTagsByType :many
SELECT * FROM tags
WHERE 'type' = $1
ORDER BY name DESC;


Only 1 SSB per Profile can be launched

Issue
When attempting to launch an SSB through the FireSSB extension from the desktop, it provides the above error message. It runs as expected when no other Firefox windows are open, or if a different profile is selected to launch the SSB in. However, launching a second SSB in that same profile would trigger the same issue, so currently a different profile would be required for each SSB, whereas we would want to run multiple SSBs in the default profile alongside each other.
System

Desktop Entry
[Desktop Entry]
Encoding=UTF-8
Version=1.0
Type=Application
Terminal=false
Exec=firefox -P firessb 'moz-extension://9747a147-d331-4e20-bdad-b51da3606593/ssb.html?url=https=-=3A=-=2F=-=2Fchat.openai.com=-=2F&name=ChatGPT&incognito=false'
Name=GPT
Icon=applications-internet

Normally multiple SSB windows can be opened with same profile and same time.
The error is popups when Firefox was started with -new-instance flag. It could be default config of Fedora. I will check the configuration of Fedora package.
Also can you try to add -new-tab or -new-window flag. It maybe solve the issue.


Bosun will have false alerts upon initalization

It is possible we end up with things like the following:

My guess is that it is because the alert rule ran before the search data had been populated. As a workaround, maybe would delay alert evaluation when bosun inits with an empty state file. This doesn't seem terribly important, but when we open source it this will be the first impression a user gets, so I think it is worth at least a workaround.

We have a wait on start now for checking unknowns, so this shouldn't be an issue.


Orange Pi Zero

Hello!
This Project support the Orange Pi Zero (http://www.orangepi.org/orangepizero/) with built-in WiFi ?
XR819, IEEE 802.11 b/g/n
Vy 73! de SQ7MRU, Pawel

Thats a really good question, any answers?

I don't see why not. It's just another network adapter.

Are there any confirmed information about working with Orange Pi Zero? I do use them, but after completing the installation and saving settings (for internal mesh node) I, as it is written in manual, rebooted the system, and for some reason the system did not boot at all. The only solution I have found is to reinstall the system...
OS: Armbian
UPD: I have done the same with my friend's Raspberry 3 and got the same result - on screen there are 4 squares (red, yellow, blue and green) and system doesn't want to boot.
Where the problem can be?

@ilyavy can you update your reply with boot logs? https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=23026

IIIRC that is a symbol for insufficient current from the power supply. Try
a beefier PSU.
On 13 June 2017 at 15:20, ilyavy<EMAIL_ADDRESS>wrote:

Are there any confirmed information about working with Orange Pi Zero? I
do use them, but after completing the installation and saving settings (for
internal mesh node) I, as it is written in manual, rebooted the system, and
for some reason the system did not boot at all. The only solution I have
found is to reinstall the system...
OS: Armbian
UPD: I have done the same with my friend's Raspberry 3 and got the same
result - on screen there are 4 squares (red, yellow, blue and green) and
system doesn't want to boot.
Where the problem can be?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/urlgrey/hsmm-pi/issues/113#issuecomment-308132025,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AALkVATXLDEKUcoJKXvX7aQVbILk4BBEks5sDprCgaJpZM4Lqevp
.

@tekstrand thank you for your quick reply. I updated my post with the logs from /var/log


Fix word boundaries on commands

'level id for a frankly unbelievable amount of doors' returns the 0th rankings overall since it includes the word 'rank'... should probably limit that to commands that explicitly include 'rank' or 'rankings'. Ditto for 'stats' and 'statistics', probably others.

I addressed this issue in a couple of ways in the commit https://github.com/edelkas/inne/commit/bcebbd8217787d703b1b4f6b3983599942837ae3. First, I separated the strictly global functions (those that are never going to reference explicit levels or players, like rankings or histories), so they will never be called accidentally by level names.
On the others, which can't be separated (because I'm using the NAME_PATTERN to filter them out), I addressed all problematic level names individually, both by adding word boundaries, and when that's not enough, by excluding other words of the level names, e.g., the level "list of inapropriate words" will no longer trigger the method "send_list".
All in all, there's very few of this cases, so the code has barely been complicated. I've tried to make it as minimal as possible too, with as few restrictions, e.g., I haven't added but the necessary word boundaries to avoid all accidents, so that the string is still matched by as many misprinted inputs as possible.


Replace :nth-child(n) with counter() in calc()

Depends on https://github.com/w3c/csswg-drafts/issues/1026


Big p2p refactor

Merge for the big-p2p-refactor branch.  All unit tests passing.  Should merge two and outstanding PR's on koinos-mq-golang.

Closes #57.


Add BaseSpec trait for plugin projects

Description
If we want to use trait objects more in the future which also bring in the initialization of properties and providers we need a good base for these factories. At the moment each plugin is creating it's own BaseSpec class. It would make sense if this is provided by this library
package com.wooga.gradle.commons

import org.gradle.api.file.ProjectLayout
import org.gradle.api.model.ObjectFactory
import org.gradle.api.provider.ProviderFactory

import javax.inject.Inject

trait BaseSpec {
    @Inject
    ProjectLayout getLayout() {
        throw new Exception("ProjectLayout is supposed to be injected here by gradle")
    }

    @Inject
    ProviderFactory getProviderFactory() {
        throw new Exception("ProviderFactory is supposed to be injected here by gradle")
    }

    @Inject
    ObjectFactory getObjects() {
        throw new Exception("ObjectFactory is supposed to be injected here by gradle")
    }
}

I agree. It will be done.


docs(sources): arras4_core readability fixup

Note this pull request was closed as an unintended side-effect of our removal of the release branch.   Please feel free to file a new pull request against the main branch, If these changes are still relevant.  Thank you!


Fix XRP block parse

Headline
Some XRP transactions can be a transaction receipt about from other coins. Like the payload bellow, this is an ETH transaction inside the XRP ledger.
Problem
We have a problem when we go to marshall the payload because we expect a string from delivered_amount field.
Solution
Create an object to marshall and add omitempty flag to the struct.
{
  "hash": "1D849E3A0041357EE373C7E17C9564F890047475492D9530B5F20A3BD6D95822",
  "ledger_index": 49841027,
  "date": "2019-09-06T01:48:32+00:00",
  "tx": {
    "TransactionType": "Payment",
    "Flags":<PHONE_NUMBER>,
    "Sequence": 292765,
    "LastLedgerSequence": 49841035,
    "Amount": {
      "value": "100000",
      "currency": "ETH",
      "issuer": "rJavT3eWaX9FubZFHtCvymJ6ZhSgJdMyNx"
    },
    "Fee": "162",
    "SendMax": "100000000000",
    "Account": "r4NT6UfELQyoS689VLye22B3SfgvpM3nHY",
    "Destination": "rJavT3eWaX9FubZFHtCvymJ6ZhSgJdMyNx"
  },
  "meta": {
    "TransactionIndex": 16,
    "DeliveredAmount": {
      "value": "533.92",
      "currency": "ETH",
      "issuer": "rJavT3eWaX9FubZFHtCvymJ6ZhSgJdMyNx"
    },
    "TransactionResult": "tesSUCCESS",
    "delivered_amount": {
      "value": "533.92",
      "currency": "ETH",
      "issuer": "rJavT3eWaX9FubZFHtCvymJ6ZhSgJdMyNx"
    }
  }
}

What? I did not know you can do that! Very awesome
We gonna show only XRP in the list correct?

No, we going return all transactions. But its simple to filter only the XRP transactions.
There is another type of transactions we can avoid, to create a destination tag, we need to do a transaction, we always show this type of transaction as well. This is a list of transactions type:
https://xrpl.org/transaction-types.html

List of XRP Transactions: http://blockatlas.trustwalletapp.com/v2/ripple/transactions/:address
List of transaction for a specific token: http://blockatlas.trustwalletapp.com/v2/ripple/transactions/:address?token=:token_id
This is a current flow, so in this case first API should only return xrp transactions, otherwise it would break iOS/Android clients

But in this case, we are using the block API to get all transactions from a block with the Blockatlas observer.

For BlockAPI you can parse all transactions, but notify only for XRP, because we don't support any tokens on xrp

done!

@madcake can you take a look at this one too

How we can change headers? Is it work for other rpc?

I did some tests here, but we need to deploy and monitoring. the first 2 lines its means we gonna accept all headers and encodings, and the third can be a problem, but if we using JSON-RPC for all calls, I think don't have problem
req.Header.Add("Accept-Encoding", "*")
req.Header.Add("Accept", "*/*")
req.Header.Add("Content-Type","application/json; charset=utf-8")


Change approach to find p2c relation dates.

General fixes.
Refactoring.

Should be run with USE_TZ = False


RD-2873 Use requests_toolbelt for file upload

No point us implementing our own version of it.

This is part of the following batch of PRs:
https://github.com/cloudify-cosmo/cloudify-common/pull/1144
https://github.com/cloudify-cosmo/cloudify-manager/pull/3826
https://github.com/cloudify-cosmo/cloudify-cli/pull/1439
https://github.com/cloudify-cosmo/cloudify-stage/pull/2171
As noted in several places, it's to avoid us having to do silly things with enco
ding of query params (and thus make the new snapshots not require more ugliness).

This isn't required any more, a change was made elsewhere to avoid breaking back compat. Closing.


DD/DDM/DMS Conversion does not work for region settings with "," decimal separator

Reported by Esri Spain via Tech Support:
The Coordinate Conversion ArcMap 1.0.1 add-in fails to populate the DMS, DD, and DDM fields if the region is set to use a comma as the decimal separator.

From the control panel > Region and language > Additional settings > Numbers tab > Decimal symbol ','> OK > OK
Download the add-in from:
http://solutions.arcgis.com/defense/templates/coordinate-conversion/#Download
Install it.
From the 'customize' menu > customize mode > commands tab > search for 'coordinate conversion'
Drag it to any existing toolbox
Open any existing map in Arcmap > click on the icon 'show coordinate conversion'
Type any coordinate or use the 'map point tool'
The DMS, DD and DDM will not display.
See video:
http://screencast-o-matic.com/watch/cDjvIgjnQ4

@topowright

As I'm working on doc for d&d I was able to verify that this works now, closing.


TF2.4/XLA assertion in Electra model when deferring compilation on GPU

Attached are gzip versions of

git.patch
dummy_p1.tfrecord
run_electra_bug.sh

Environment:

Google TF-2.4 container: tensorflow/tensorflow:2.4.0-gpu
single GV100 32GB

Reproduction steps:

git clone git@github.com:NVIDIA/DeepLearningExamples.git
cd DeepLearningExamples/TensorFlow2/LanguageModeling/ELECTRA/
copy the attached git.patch file to ./
cp the attached run_electra_bug file to ./scripts
cp the attached dummy_p1.tfrecord file to ./data
git apply git.patch
bash scripts/docker/build.sh
bash scripts/docker/launch.sh
TF_XLA_FLAGS=--tf_xla_always_defer_compilation=true bash scripts/run_electra_bug.sh 1 6e-3 amp 1  (run this within the docker container started in 8).

The resulting error:
0]<stderr>:Traceback (most recent call last):
[0]<stderr>:  File "/workspace/electra/run_pretraining.py", line 493, in <module>
[0]<stderr>:    args = main(start_time)
[0]<stderr>:  File "/workspace/electra/run_pretraining.py", line 427, in main
[0]<stderr>:    local_step==1, take_step=local_step % args.gradient_accumulation_steps == 0)
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 828, in __call__
[0]<stderr>:    result = self._call(*args, **kwds)
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py", line 888, in _call
[0]<stderr>:    return self._stateless_fn(*args, **kwds)
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 2943, in __call__
[0]<stderr>:    filtered_flat_args, captured_inputs=graph_function.captured_inputs)  # pylint: disable=protected-access
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 1919, in _call_flat
[0]<stderr>:    ctx, args, cancellation_manager=cancellation_manager))
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py", line 560, in call
[0]<stderr>:    ctx=ctx)
[0]<stderr>:  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py", line 60, in quick_execute
[0]<stderr>:    inputs, attrs, num_outputs)
[0]<stderr>:tensorflow.python.framework.errors_impl.CancelledError:  [_Derived_]RecvAsync is cancelled.
[0]<stderr>:	 [[{{node cluster_11_1/merge_oidx_69/_148}}]] [Op:__inference_train_one_step_88467]
[0]<stderr>:
[0]<stderr>:Function call stack:
[0]<stderr>:train_one_step
[0]<stderr>:
Process 0 exit with status code 1.
Traceback (most recent call last):
  File "/usr/local/bin/horovodrun", line 8, in <module>
    sys.exit(run_commandline())
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 768, in run_commandline
    _run(args)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 758, in _run
    return _run_static(args)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 615, in _run_static
    _launch_job(args, settings, nics, command)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 731, in _launch_job
    args.verbose)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 704, in run_controller
    gloo_run()
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/launch.py", line 720, in gloo_run_fn
    gloo_run(settings, nics, env, driver_ip, command)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/gloo_run.py", line 284, in gloo_run
    launch_gloo(command, exec_command, settings, nics, env, server_ip)
  File "/usr/local/lib/python3.6/dist-packages/horovod/runner/gloo_run.py", line 271, in launch_gloo
    .format(name=name, code=exit_code))

[
git.patch.gz
dummy_p1.tfrecord.gz
run_electra_bug.sh.gz
](url)

[
git.patch.gz
dummy_p1.tfrecord.gz
run_electra_bug.sh.gz
](url)

@bas-aarts,
The code provided is fairly complex hence it would be difficult for us to pinpoint the issue. Could you please get the example down to the simplest possible repro? That will allow us to determine the source of the issue easily. Thanks!

@bas-aarts,
The code provided is fairly complex hence it would be difficult for us to pinpoint the issue. Could you please get the example down to the simplest possible repro? That will allow us to determine the source of the issue easily. Thanks!

I was able to reproduce this.
When I look at the entire log though, I see an OOM.  Do you see that as well?  If yes, it is possible that the OOM is the root cause and the _Recv failed because it was waiting on a _Send that failed because of the OOM.  Can you try adjusting the batch size etc. to make the OOM disappear and check if the failed recv error also goes away?

I was able to reproduce this.
When I look at the entire log though, I see an OOM.  Do you see that as well?  If yes, it is possible that the OOM is the root cause and the _Recv failed because it was waiting on a _Send that failed because of the OOM.  Can you try adjusting the batch size etc. to make the OOM disappear and check if the failed recv error also goes away?

@sanjoy, shooting from the hip here, but can you try running on 1 GPU only?
I reproduced on 1 GPU only (tried again just now on GV100 32GB), and see no OOM.
Perhaps multiple GPUs results in more allocation per GPU
try running with:
TF_XLA_FLAGS=--tf_xla_always_defer_compilation=true bash scripts/run_electra_bug.sh 1 6e-3 amp 1

@sanjoy, shooting from the hip here, but can you try running on 1 GPU only?
I reproduced on 1 GPU only (tried again just now on GV100 32GB), and see no OOM.
Perhaps multiple GPUs results in more allocation per GPU
try running with:
TF_XLA_FLAGS=--tf_xla_always_defer_compilation=true bash scripts/run_electra_bug.sh 1 6e-3 amp 1

The

I actually tried, but was not able to reduce significantly. I therefore choose to use the model as a repro.

The

I actually tried, but was not able to reduce significantly. I therefore choose to use the model as a repro.

Bas, is there a way to reduce the batch size so that it fits in a 16G P100 or V100?  I don't have access to a 32G V100.

Bas, is there a way to reduce the batch size so that it fits in a 16G P100 or V100?  I don't have access to a 32G V100.

With the last  command line, BS=1

With the last  command line, BS=1

Bas, this is the full log with 1GPU and the command line you mentioned in https://github.com/tensorflow/tensorflow/issues/46419#issuecomment-764001365. In the log I see many lines like:
[0]<stderr>:2021-01-26 05:17:39.245145: W tensorflow/core/framework/op_kernel.cc:1763] OP_REQUIRES failed at resource_variable_ops.cc:536 : Not found: Resource localhost/_AnonymousVar837/N10tensorflow3VarE does not exist.

preceding the RecvAsync failure.  So I suspect these errors are causing a previous Send to fail, resulting in the later failed Recv.  Does this sound plausible?

Bas, this is the full log with 1GPU and the command line you mentioned in https://github.com/tensorflow/tensorflow/issues/46419#issuecomment-764001365. In the log I see many lines like:
[0]<stderr>:2021-01-26 05:17:39.245145: W tensorflow/core/framework/op_kernel.cc:1763] OP_REQUIRES failed at resource_variable_ops.cc:536 : Not found: Resource localhost/_AnonymousVar837/N10tensorflow3VarE does not exist.

preceding the RecvAsync failure.  So I suspect these errors are causing a previous Send to fail, resulting in the later failed Recv.  Does this sound plausible?

I'm certain the warnings have something to do with the problem.  Since these are no errors, I do not know how serious the issue is.

I'm certain the warnings have something to do with the problem.  Since these are no errors, I do not know how serious the issue is.

Any updates on this? perhaps even an ETA?

Bas,
Sorry for the delay, after https://github.com/tensorflow/tensorflow/issues/46419#issuecomment-767343956 I assumed that you're going to look into this further.  Looks like that was a mis-parsing on my part.
I strongly suspect that some Grappler-esque pass is running after auto-clustering and DCE'ing an XLA cluster.  The following change seems to fix the symptom at least (log):
diff --git a/tensorflow/compiler/jit/build_xla_ops_pass.cc b/tensorflow/compiler/jit/build_xla_ops_pass.cc
index a340b9d3f45..e7c0c10a225 100644
--- a/tensorflow/compiler/jit/build_xla_ops_pass.cc
+++ b/tensorflow/compiler/jit/build_xla_ops_pass.cc
@@ -309,9 +309,9 @@ xla::StatusOr<Node*> ReplaceFunctionCallWithPartitionedCall(
     }
   }

-  ops::PartitionedCall call(
-      root.WithOpName("partitioned_call"), args, n->output_types(), func,
-      ops::PartitionedCall::Attrs{}.ConfigProto(config_string));
+  ops::StatefulPartitionedCall call(
+      root.WithOpName("stateful_partitioned_call"), args, n->output_types(),
+      func, ops::StatefulPartitionedCall::Attrs{}.ConfigProto(config_string));

   for (const Edge* e : n->in_edges()) {
     if (e->IsControlEdge()) {

So I think the correct solution is to use StatefulPartitionedCall when the XLA cluster has any resource variable operations.  But before we do that, I'd prefer confirm that this theory is correct -- would you be able to put some time into verifying this theory?  For this theory to be correct, there has to be a DCE-like pass that runs after build_xla_ops_pass that deletes non-stateful partitioned call ops.  This pass is likely being run from here.
@cheshire and @nnigania also ran into a variant of this issue IIRC.

I'll take a look @sanjoy

@bas-aarts As discussed, there is the patch I've used.
In file def_function.py, after inserting _noinline=True, add

attributes.update(_grappler_do_not_remove=True) (line 658)

Inside functional_ops.py, at line 1223, add:
  if grappler_do_not_remove_attr in f.definition.attr:
    op_attrs[grappler_do_not_remove_attr] = f.definition.attr[
      grappler_do_not_remove_attr]

where grappler_do_not_remove_attr = '_grappler_do_not_remove

Undoing auto-close.

@cheshire, would you mind attaching a git patch instead? want to make sure I do the right thing. thanks.

hi Bas,
I don't have a git patch, as this change is internal. Do you think you
could try it and maybe show the patch you create to double check?
George
On Wed, Feb 17, 2021 at 2:22 PM Bas Aarts<EMAIL_ADDRESS>wrote:

@cheshire https://github.com/cheshire, would you mind attaching a git
patch instead? want to make sure I do the right thing. thanks.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/46419#issuecomment-780893717,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AACVGH2AWJLHSVVMPHBVEKDS7Q6S3ANCNFSM4WB547YA
.

Hi George,
with the following diff, the error is still there:
diff --git a/tensorflow/python/eager/def_function.py b/tensorflow/python/eager/def_function.py
index da19acad243..475fa810433 100644
--- a/tensorflow/python/eager/def_function.py
+++ b/tensorflow/python/eager/def_function.py
@@ -655,6 +655,7 @@ class Function(object):
attributes.update(_XlaMustCompile=bool(self._jit_compile))
if self._jit_compile:
attributes.update(_noinline=True)

   attributes.update(_grappler_do_not_remove=True)

if not attributes:
attributes = None
return function_lib.defun_with_attributes(
diff --git a/tensorflow/python/ops/functional_ops.py b/tensorflow/python/ops/functional_ops.py
index bdd20cda991..2caae0dc24e 100644
--- a/tensorflow/python/ops/functional_ops.py
+++ b/tensorflow/python/ops/functional_ops.py
@@ -1218,6 +1218,9 @@ def partitioned_call(args,
}
if xla_compile_attr in f.definition.attr:
op_attrs[xla_compile_attr] = f.definition.attr[xla_compile_attr]
grappler_do_not_remove_attr = "_grappler_do_not_remove"
if grappler_do_not_remove_attr in f.definition.attr:
op_attrs[grappler_do_not_remove_attr] = f.definition.attr[grappler_do_not_remove_attr]
op = graph.create_op(op_name, args, tout, name=op_name, attrs=op_attrs)
outputs = op.outputs
if hasattr(f, "graph"):

Hi George.
with the following diff the original error is still there
diff --git a/tensorflow/python/eager/def_function.py b/tensorflow/python/eager/def_function.py
index da19acad243..475fa810433 100644
--- a/tensorflow/python/eager/def_function.py
+++ b/tensorflow/python/eager/def_function.py
@@ -655,6 +655,7 @@ class Function(object):
       attributes.update(_XlaMustCompile=bool(self._jit_compile))
       if self._jit_compile:
         attributes.update(_noinline=True)
+        attributes.update(_grappler_do_not_remove=True)
     if not attributes:
       attributes = None
     return function_lib.defun_with_attributes(
diff --git a/tensorflow/python/ops/functional_ops.py b/tensorflow/python/ops/functional_ops.py
index bdd20cda991..2caae0dc24e 100644
--- a/tensorflow/python/ops/functional_ops.py
+++ b/tensorflow/python/ops/functional_ops.py
@@ -1218,6 +1218,9 @@ def partitioned_call(args,
   }
   if xla_compile_attr in f.definition.attr:
     op_attrs[xla_compile_attr] = f.definition.attr[xla_compile_attr]
+  grappler_do_not_remove_attr = "_grappler_do_not_remove"
+  if grappler_do_not_remove_attr in f.definition.attr:
+    op_attrs[grappler_do_not_remove_attr] = f.definition.attr[grappler_do_not_remove_attr]
   op = graph.create_op(op_name, args, tout, name=op_name, attrs=op_attrs)
   outputs = op.outputs
   if hasattr(f, "graph"):

I have confirmed that @sanjoy's proposed change does not trigger the error.

@cheshire , can you please let me know if the diff above is the one you expected to work? Thanks

@bas-aarts yes. To clarify: I do not know whether it solves your problem or not. It solved one problem for me, I'm not sure whether you have the same one.


Multiple update submodules when discarding multiple files at once

2 refreshes are called

Gittyup v1.1.1 - Donnerstag, 17. November 2022 - aea75e68


Change from Glide to Dep and build for Windows.

When I tried to build it for Windows it was failing so I made the following changes.

Updated makefile for dep and Windows
Changed from Glide to Dep.
Upgraded vendor sources
WinRM hack in vendor included until https://github.com/masterzen/winrm/pull/86 is merged

Hi @taliesins, thank you for the PR. Thank you for the PR. Unfortunately, I'm not able to take this in for the following reason:
While working on releases (sparked by https://github.com/radekg/terraform-provisioner-ansible/issues/55), I've made upgrades to the code, such that the user of the provisioner should not need to build anything anymore.
Releases are available via GitHub releases and Terraform Module Registry.
I appreciate that some additional updates from your PR might be swept under the carpet by rejecting this PR. If that's the case and you believe they could be beneficial to others, please open a separate PR.


Add compose ps flags

Add missing flags for the compose ps command.
Fixes #2502

Please squash commits


Fixed adminrouter package installation issue from the code earlier code changes

Changes included:

Fixed the MSI launch command parameters error, which was causing the MSI package run to fail
Added Outbound rules for updating url fields in the response packets

Looks good to me.

@yakman2020, @paulcallen We need an approve to merge this


Remove deprecated learn.ops

These ops are deprecated and users need to use ops in contrib.layers instead.

@tensorflow-jenkins Test this please

@tensorflow-jenkins Test this please


Add a "dry run" parameter

Many cli tools have a "dry run" option (sometimes -n) which does everything except executing the modifications. This should be a very small change, but useful for the paranoid admin.

done, thanks a lot for your proposal. This makes a lot of sense to have. An initial version was committed with #67 and will be included with the next RC and with the greatest 1.0 version :-)


Failing to add packages from Atom

I'm quite new to both atom and paket, so it might be a beginner mistake, but I guess we don't want them either.
I installed atom, the fsharp and paket plugin for atom. When I try to run Add Nuget package I get:
Paket failed with:
	    Error in paket.dependencies line 3
  could not retrieve nuget package from

When I run paket install from command line it seems to be working just fine though. Any ideas of what's wrong or anything I could do to help figure it out?

Could you please provide the paket.dependencies file?

And what package did you try to add

The paket.dependencies file:
source https://nuget.org/api/v2

nuget FAKE
nuget Suave
nuget DotLiquid
nuget FSharp.Data
nuget FSharp.Compiler.Service
nuget FSharp.Data.SqlClient

I started with a project from Petricek and just wanted to add FSharp.Data.SqlClient, the FSharp.Data.SqlClient in the file was added manually and after that it worked to run paket install.
How is the plugin supposed to work? Do I need to install paket separately and at it to the path or should the plugin handle everything? I'm on Windows btw.

The atom plugin should handle everything. It downloads latest paket.exe on
install. /cc @krysztof_cieslak
The paket.dependencies file:
source https://nuget.org/api/v2
nuget FAKE
nuget Suave
nuget DotLiquid
nuget FSharp.Data
nuget FSharp.Compiler.Service
nuget FSharp.Data.SqlClient
I started with a project from Petricek and just wanted to add
FSharp.Data.SqlClient, the FSharp.Data.SqlClient in the file was added
manually and after that it worked to run paket install.
How is the plugin supposed to work? Do I need to install paket separately
and at it to the path or should the plugin handle everything? I'm on
Windows btw.
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/888#issuecomment-113902507.

I think I see the problem - the Paket for some reason (which I don't know yet) doesn't returns list of plugins - only element visible on should-be-searched-packages list is empty element ( displaying it is definitely the bug in plugin) and when user press Enter it's using chosen element from list (so in this case it's empty element) and we invoke Paket with wrong arguments.
So it's definitely plugin bug, but still for some reason Paket is not returning search results.

Hmm... looks like I get search results from command line. Will investigate bit more

@forki: When I fire paket in my local project location (just .paket\paket.exe) i get normal search results. When I try to run it using full path to paket atom plugin (i.e. C:\Users\Krzysztof\.atom\packages\Paket\bin\paket.exe ) I get empty result form it. Any idea?

Maybe it doesn't find the right paket.dependencies to look up the nuget
feed urls?
On Jun 21, 2015 4:10 PM, "Krzysztof Cieślak"<EMAIL_ADDRESS>wrote:

@forki https://github.com/forki: When I fire paket in my local project
location (just .paket\paket.exe) i get normal search results. When I try
to run it using full path to paket atom plugin (i.e.
C:\Users\Krzysztof.atom\packages\Paket\bin\paket.exe ) I get empty
result form it. Any idea?
—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/888#issuecomment-113905792.

I'm starting Paket in both cases from same directory
Other thing is that I'm kinda sure it WAS working yesterday or 2 days ago - don't remember when i lasted used Paket plugin

Ok will try to reproduce tomorrow.
On Jun 21, 2015 4:37 PM, "Krzysztof Cieślak"<EMAIL_ADDRESS>wrote:

I'm starting Paket in both cases from same directory
Other thing is that I'm kinda sure it WAS working yesterday or 2 days ago

don't remember when i lasted used Paket plugin

—
Reply to this email directly or view it on GitHub
https://github.com/fsprojects/Paket/issues/888#issuecomment-113909689.

yes I broke find-packages when I tried to fix find-package-versions - see https://github.com/fsprojects/Paket/issues/886#issuecomment-114022544
sorry about that

I've also added in Paket.Atom some better behavior if Paket for any reason returns empty result - same bug could occurred if someone gave text which could not be found by nuget. Should be better now.

Very good. Thanks


import span to speed up

Data copy is bad for performance. I import span(c++20) for speeding up data access.

https://en.cppreference.com/w/cpp/language/extending_std
It is undefined behavior to add declarations or definitions to namespace std or to any namespace nested within std, with a few exceptions noted below
I don't think defining std::span satisfies any of the exceptions. (Please point it out if I am wrong here.) It may cause name conflict with some compilers.
With that being said, I do agree to avoid data copy to improve performance. One way is to define span in another namespace, e.g. cppflow::span and uses std::span (typedef) when possible. The other way would be defining access functions on cppflow::tensor, e.g. operator[] and iterators.

Yes, you are right. std::span defined in c++20. Maybe we can use the standard span if using compiler supported c++20.

https://en.cppreference.com/w/cpp/utility/feature_test
https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html
Maybe check __cpp_lib_span, and use std::span if it is available, otherwise
use the self defined version.
On Wed, Nov 4, 2020, 00:33 Allen<EMAIL_ADDRESS>wrote:

Yes, you are right. std::span defined in c++20. Maybe we can use the
standard span if using compiler supported c++20.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/serizba/cppflow/pull/68#issuecomment-721521263, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AALDTHSCRT3UCRFAXPJQ7D3SODRSRANCNFSM4TJQ7XVA
.

https://en.cppreference.com/w/cpp/utility/feature_test https://gcc.gnu.org/onlinedocs/libstdc++/manual/status.html Maybe check __cpp_lib_span, and use std::span if it is available, otherwise use the self defined version.
…
On Wed, Nov 4, 2020, 00:33 Allen @.***> wrote: Yes, you are right. std::span defined in c++20. Maybe we can use the standard span if using compiler supported c++20. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#68 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALDTHSCRT3UCRFAXPJQ7D3SODRSRANCNFSM4TJQ7XVA .

Okay, I have updated my code to support c++20 feature :)

Add test for span in c++17 and c++20 and test passed.

You're really serious. LOL. I got an `std::runtime_error' undefined when i remove . Fixed it by the way.

You're really serious. LOL. I got an `std::runtime_error' undefined when i remove . Fixed it by the way.

LOL. Thanks for the fix BTW. I am really looking forward to using this lib in my code, so I just want to make it pretty and clean. Thanks for bearing with me. We still need to look at the behavior of res_tensor though...

I guess I need to go through the source code to see how TF manages memory
internally. My first speculation is that TF_Tensor is RefCounted, so we
should be safe to delete it after obtaining an eager handle. (pure
speculation)
On Thu, Nov 5, 2020, 2:52 AM Allen<EMAIL_ADDRESS>wrote:

@ivanallen commented on this pull request.
In include/cppflow/tensor.h
https://github.com/serizba/cppflow/pull/68#discussion_r517849884:

@@ -205,7 +206,7 @@ namespace cppflow {

     // Convert to correct type
     const auto T_data = static_cast<T*>(raw_data);

Yeah, you are right! It indeed confuses people. Actually, res_tensor and
this->tf_tensor hold the same buffer, but sometimes res_tensor uses a new
buffer.
I think we should use the raw data of this->tf_tensor but not res_tensor.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/serizba/cppflow/pull/68#discussion_r517849884, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AALDTHXJAUCQZ6CGTWNBJ4DSOJKSHANCNFSM4TJQ7XVA
.

I guess I need to go through the source code to see how TF manages memory internally. My first speculation is that TF_Tensor is RefCounted, so we should be safe to delete it after obtaining an eager handle. (pure speculation)
…
On Thu, Nov 5, 2020, 2:52 AM Allen @.> wrote: @.* commented on this pull request. ------------------------------ In include/cppflow/tensor.h <#68 (comment)>: > @@ -205,7 +206,7 @@ namespace cppflow { // Convert to correct type const auto T_data = static_cast<T*>(raw_data); Yeah, you are right! It indeed confuses people. Actually, res_tensor and this->tf_tensor hold the same buffer, but sometimes res_tensor uses a new buffer. I think we should use the raw data of this->tf_tensor but not res_tensor. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#68 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALDTHXJAUCQZ6CGTWNBJ4DSOJKSHANCNFSM4TJQ7XVA .

I looked tensorflow source:
TF_Tensor* TF_TensorFromTensor(const tensorflow::Tensor& src,
                               TF_Status* status) {
  if (src.dtype() == DT_RESOURCE) {
    // ...
    // This branch is unsafe.
    const string str = src.scalar<ResourceHandle>()().SerializeAsString();
    TF_Tensor* t = TF_AllocateTensor(TF_RESOURCE, {}, 0, str.size());
    std::memcpy(TF_TensorData(t), str.c_str(), str.size());
    return t;
  }
  if (src.dtype() != DT_STRING) {
    // This branch is safe.
    TensorBuffer* buf = TensorCApi::Buffer(src);
    buf->Ref();
    return new TF_Tensor{static_cast<TF_DataType>(src.dtype()), src.shape(),
                         buf};
  }

So the result is up to dtype.

if (src.dtype() == DT_RESOURCE) {
    // ...
    // This branch is unsafe.
    const string str = src.scalar<ResourceHandle>()().SerializeAsString();
    TF_Tensor* t = TF_AllocateTensor(TF_RESOURCE, {}, 0, str.size());
    std::memcpy(TF_TensorData(t), str.c_str(), str.size());
    return t;
  }

Maybe it leads to a memory leak.

if (src.dtype() == DT_RESOURCE) {
    // ...
    // This branch is unsafe.
    const string str = src.scalar<ResourceHandle>()().SerializeAsString();
    TF_Tensor* t = TF_AllocateTensor(TF_RESOURCE, {}, 0, str.size());
    std::memcpy(TF_TensorData(t), str.c_str(), str.size());
    return t;
  }

Maybe it leads to a memory leak.

Yes, I agree res_tensor leaks in our original code. Thanks for spotting it!

@ivanallen @serizba
I think it is safe to store the resolved TF_Tensor* res_tensor into this->tf_tensor in tensor::get_data(). So we can avoid the memory leak, and also make sure res_tensor won't be deleted while we are accessing the returned data through span.
What happens under the hood is the following.
(1) When we call TFE_NewTensorHandle, a new copy of the source tensor is created and moved (std::move) into a new TensorHandle. This makes sure the source tensor can be deleted after a TensorHandle is obtained.
(2) When we call TFE_TensorHandleResolve, the underlying TensorHandle::Resolve() is invoked. Depending on the context, a new concrete tensor may be copied from a device or a local copy/mirror is returned. We can just store this tensor into this->tf_tensor. In the original code, we definitely need to delete res_tensor at the end of tensor::get_data() to avoid memory leak.
@ivanallen For TF_TensorFromTensor, I think the dtype() dependent behavior is due to their different data representations, please see here. I don't know DT_RESOURCE, but DT_STRING may contain pointers to heap allocated buffer. Other data type should be just plain memory buffer.

I use AddressSanitizer detected some memory leak and fixed it. The method get_data really has a memory leak. But we can directly use this->tf_tensor obtaining raw data. It should be safe and no memory leak.

@ivanallen I noticed the leak messages with gcc's sanitizer before but did not get the time to trace it down. I think the bug fix should be in a separate PR for easier testing.
We cannot get data from this->tf_tensor, and calling TFE_TensorHandleResolve is a must. After TFE_TensorHandle is obtained, there is no guarantee that this->tf_tensor and this->tfe_handle shares the same memory, e.g. the data can be placed on GPU and a new fetch is needed after some computations. All the eager ops are using this->tfe_handle. Please refer to my previous comment for more details. In addition, some of the constructor of tensor class (tensor::tensor(TFE_TensorHandle* handle)) does not even set a valid this->tf_tensor
What I meant was to change
auto res_tensor = TFE_TensorHandleResolve(this->tfe_handle.get(), context::get_status());
to
this->tf_tensor = TFE_TensorHandleResolve(this->tfe_handle.get(), context::get_status());
There should be no data corruption/leaking issue and all lifetime is managed correctly. However, we still need more testing on this.

You know, the get_data method's type is const-qualified, so modify this->tf_tensor seems unreasonable unless you redesign the method sematic to non-const or letthis->tf_tensor mutable.

But we really don't have other choices to keep its ownership, and resolving
the tensor does change the underlying data, though not the pointer itself.
An alternative is to return std::span.
On Sat, Nov 7, 2020, 02:09 Allen<EMAIL_ADDRESS>wrote:

You know, the get_data method's type is const-qualified, so modify
this->tf_tensor seems unreasonable unless you redesign the method sematic
to non-const or letthis->tf_tensor mutable.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/serizba/cppflow/pull/68#issuecomment-723402793, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AALDTHUZTMRPW3EBKGV6BPDSOTXDFANCNFSM4TJQ7XVA
.

@ivanallen Why is this closed? I think it is still worth discussing at least.
@serizba I would also like to know your opinion on this before I started to test it.

@ivanallen Why is this closed? I think it is still worth discussing at least.
@serizba I would also like to know your opinion on this before I started to test it.

sorry, I just update cppflow2 in my github. I will reopen this PR.

@ivanallen @ljn917
Sorry for not attending this PR before.
Regarding std::span sounds like a good idea, but what about not c++20 compilers?
For the res_tensor thing, I also think using a new res_tensor was a bad idea from my side, using this->tf_tensor seems more reasonable.
The PR is interesting, so no need for closing it :)

@serizba

Regarding std::span sounds like a good idea, but what about not c++20 compilers?

We are adding a standard compatible span.h for pre-C++20 compilers.

For the res_tensor thing, I also think using a new res_tensor was a bad idea from my side, using this->tf_tensor seems more reasonable.

We were discussing converting this->tf_tensor to a private member which would serve as a local cache of the resolved tensor. Perhaps, we also need to wrap TFE_TensorHandleResolve in a get method.

@ivanallen Finally, we are ready to work on this PR :)
First, could you clean up this PR a bit? Include the following:

Rebase onto the updated cppflow2 branch. This should address the change in include/cppflow/context.h and include/cppflow/tensor.h
Remove changes to the following files as they are not relevant:
examples/eager_op_multithread/CMakeLists.txt
include/cppflow/model.h

After that, I may push some tests to your branch.

@ivanallen Finally, we are ready to work on this PR :)
First, could you clean up this PR a bit? Include the following:

Rebase onto the updated cppflow2 branch. This should address the change in include/cppflow/context.h and include/cppflow/tensor.h
Remove changes to the following files as they are not relevant:
examples/eager_op_multithread/CMakeLists.txt
include/cppflow/model.h

After that, I may push some tests to your branch.

Great! I have rebased my code.

@ivanallen Thanks a lot, but I think we have more changes to make...

The point (2) above has not been addressed, i.e. remove changes to the following files as they are not relevant:
examples/eager_op_multithread/CMakeLists.txt
include/cppflow/model.h
Those changes can still be seen at https://github.com/serizba/cppflow/pull/68/files

Make a directory named third_party in the include directory, and place the orignial Tristan Brindle's span implementation into this directory, i.e. the unmodified version. In this way, we can separate the third party libs and update them easier.

Create a header named compat_span.h in the include directory, and put our switch logic here. Macros, e.g. TCB_SPAN_NAMESPACE_NAME should also be defined here.

Define TCB_SPAN_NAMESPACE_NAME as cppflow::tcb to avoid namespace pollution, e.g. TCB_SPAN_NAMESPACE_NAME::byte.

As we discussed before, __cpp_lib_span is only defined in <version> or <span> header. We need to use __has_include feature test macro.

Use cppflow::span<const T> as the return type of tensor::get_data() function.

I believe you can also mark all the previous comments as resolved because most of them will be outdated.

The objective of this PR is to avoid copying the data when retrieving the data with get_data. But, isn't #202 a simpler way of achieving this, and thus avoiding also the inconveniences of std::vector and std::span. Besides, if you need a std::span later on, you can also use the obtained raw_data for this.
What do you think?

In my opinion, they are just two different styles of doing it, ie. the C style and the C++ style. The efficiency/convenience of getting the raw data has improved a lot since this PR, so I don't think its priority is as high as before. In addition, the key issue of this PR is importing a third party std::span. To be honest, I am not a big fan of it. I would rather wait until C++20 becomes more accessible.

Yes, that's also my impression. Perhaps for the moment I will include this with a precompiler directive when the C++20 is enabled. And I won't include the span implementation.


For base_task, task.run() will work with context if no structure is provided.

[x] I have read and agree to the contributing guidelines.

Is your feature request related to a problem? Please describe.
prompt_task = PromptTask(
    "Respond to the user's following question 'who are you' in the language '{{preferred_language}}' and tone '{{tone}}'.",
    context={"preferred_language": "ENGLISH", "tone": "PLAYFUL"},
)

if just create a task like this, the context will not filled in when prompt_task.run().
Describe the solution you'd like
the context for prompt_task work without a  structure.
in base_task.py
    @property
    def full_context(self) -> dict[str, Any]:
        if self.structure:
            structure_context = self.structure.context(self)

            structure_context.update(self.context)

            return structure_context
        else:
            return self.context

Describe alternatives you've considered
None
Additional context
None

Are you able to provide a fully reproducible example? With this example, I run into #1155 first:
from griptape.tasks import PromptTask

prompt_task = PromptTask(
    "Respond to the user's following question 'who are you' in the language '{{preferred_language}}' and tone '{{tone}}'.",
    context={"preferred_language": "ENGLISH", "tone": "PLAYFUL"},
)

prompt_task.run()

Traceback (most recent call last):
  File "/Users/collindutter/Documents/griptape/griptape-playground/src/issues/1154.py", line 8, in <module>
    prompt_task.run()
  File "/Users/collindutter/Documents/griptape/griptape/griptape/tasks/prompt_task.py", line 81, in run
    message = self.prompt_driver.run(self.prompt_stack)
                                     ^^^^^^^^^^^^^^^^^
  File "/Users/collindutter/Documents/griptape/griptape/griptape/tasks/prompt_task.py", line 48, in prompt_stack
    memory = self.structure.conversation_memory
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'conversation_memory'

Thanks!

Could not because I also run into the error, so i fix the exception myself and continued.
I just want to find the inner working of Task, if it is not supported to work alone, I could use a structure.


Multipart/form_Data using  Reqwest Async not working

Hello.
I am making a post request  that is sending a form_data but  the data is not uploaded.
The upload is done in two phase. In the First Phase am sending this data
                     let form_data = Form::new()
                    .text("upload_session_id", "id")
                    .text("upload_phase", "phase")
                    .text("access_token", "token");

while the second phase  involve also attaching  a chunk file.
Steps to replicate the issue.
   use reqwest::multipart::{Form, Part};
     use std::fs::File;
     use std::io;
     use std::io::prelude::*;

  // creating first phase form_data

let form_data = Form::new()
                 .text("file_size", "size")
                 .text(" upload_phase", "start")
                 .text("access_token", "token);

To create second phase form_data  with chunk file
//handle file
  let mut file = File::open(video_params.file_path.clone()).unwrap();
        let mut buffer = Vec::new();
        file.read_to_end(&mut buffer).unwrap(); // read all file
        let part = Part::bytes(buffer);

        let form_data = Form::new()
                    .text("start_offset", start_offset.clone())
                    .part(
                        "part",
                        part
                    )

Sending the request
         let resp =  Client::new().post(url)
            .multipart(form_data)
            .send()
            .await

The request was sent but no data  was uploaded.
But   when i used  form instead of the multipart   the first phase  data  was uploaded  as  shown below  but i cant use this approach to sending the chunk file.
  let params = [("upload_phase", "start"), ("file_size", file_size.clone())];

       let resp = Client::new()
          .post(url)
          .form(&params)
          .send()
          .await;

From the documentation , the approach   as shown above  with the multipart  should work but is not working.
What is the best approach to  sending  a request with a form_data  that include a file  in Reqwest Async  or  is something  wrong with what am doing ?
Thank  for you

No data at all was uploaded? Or just some of it? Perhaps you mean that "first phase" isn't sent, only file part? It sounds like you're creating one FormData, and then creating a second with the same name, and so the first one is forgotten.
What if you changed the second part to:
formdata = formdata
    .text(name, val)
    .part("part", part);

This would add the parts to the same FormData.

The first phase upload which does not include file or Part   as show

                     let form_data = Form::new()
                    .text("upload_session_id", "id")
                    .text("upload_phase", "phase")

                     let resp =  Client::new().post(url)
            .multipart(form_data)
            .send()
            .await

The response  i got from the server is  that it  can´t upload file but when i use   form  as shown below  in works ( just the  first phase)
let params = [("upload_phase", "start"), ("file_size", "file_size",("upload_session_id", "id"))];

     let resp = Client::new()
        .post(url)
        .form(&params)
        .send()
        .await;

It seems like the issue is on the multipart method upload since no form of data was uploaded successfully using the multipart.

The multipart method in reqwest does work properly, it has seen extensive use. It's possible the server expects different field names, values, combinations of fields, or who knows what. I don't believe this is an issue with reqwest.

The multipart method in reqwest does work properly, it has seen extensive use. It's possible the server expects different field names, values, combinations of fields, or who knows what. I don't believe this is an issue with reqwest.

I have also considered those as a possible cause. To rule out those, i decided to make the request with the same data and field name using  form instead of the   multipart which works.  I have done alot of testing to ensure that am not missing anything in terms of  filed_name, expected data, etc.

Do you have an example with multipart you could share or point to where i can see it so i can compare.
I will really appreciate it.
Thanks

Hello @seanmonstar , this  is the data/format the server is expecting. ( all other data  show has been encoded as  in the url as query parameters  so am only sending the chunk file
 curl -X POST \
 "https://graph-video.facebook.com/v12.0/1755847768034402/videos"  \
 -F "upload_phase=transfer" \
 -F "upload_session_id=99999" \
 -F "access_token=EAADI..." \
 -F "start_offset=10485760" \
 -F<EMAIL_ADDRESS>
I also printed the form_data  after adding the Part  and this is what i got
{ boundary: "2ff62a4e7e1260eb-01097f22067c50a8-7edb935c7e64fdfa-690caeeec0e0c575", parts: [("video_file_chunk", Part { value: Body, mime: None, file_name: None, headers: {} })] }

Looking at the result of the printed  form_data  am not sure if the  expected format is the same. what do you think ?
I also made a different request  that involve  uploading the whole file directly  but i got a error saying that the uploaded file is in  wrong format .
I have also test these method and file with other client.
I am uploading to facebook  documentation


Change operator default port to 443.

The operator service should serve on 443 as this is the more proper TLS port over the current 80. The injector has been updated to now inject port 443 to sidecars. Port 80 is still being served for backwards compatibility, but should be removed in v1.14.

/version-skew-test

/test-version-skew

/test-version-skew

The operator service should serve on 443 as this is the more proper TLS port over the current 80. The injector has been updated to now inject port 443 to sidecars. Port 80 is still being served for backwards compatibility, but should be removed in v1.14.

It should ideally be only removed in 1.15, 2 releases after deprecation.

@mukundansundar It was my understanding that this was not the case for non-API related control plane configuration options. For example we do not do this for removing CLI flags.


The suggested installation instructions might not properly merge old files!

The installation for loop removed my old .zshrc commands :(. I don't know much about zsh scripting, so I can't see why the files weren't merged properly, but please add a loop that makes backups!

@ivomac, thanks for pointing this out.


Error tweaks

Some deduplication and message tweaks

Should be better now.

Thanks a lot @pydsigner !

I'm checking quickly why flake8 is complaining.