Vim9: 'gdefault' should be ignored

Describe the bug
In Vim9 script, 'gdefault' is not ignored; maybe it should.
To Reproduce
Run this shell command:
vim -es -Nu NONE -S <(cat <<'EOF'
    vim9script
    set gdefault
    setline(1, 'aaa xxx aaa xxx')
    s/xxx/bbb/
    :%p
    qa!
EOF
)

It outputs:
aaa bbb aaa bbb

Expected behavior
It outputs:
aaa bbb aaa xxx

Environment

Vim version: 8.2 Included patches: 1-2165
OS: Ubuntu 16.04.7 LTS
Terminal: xterm(362)

Additional context
:h 'gd' says that this option is deprecated and might break plugins which are not aware of its existence:
https://github.com/vim/vim/blob/8f22f5c3aa0aa96e3843a57f29405625d8514c74/runtime/doc/options.txt#L3570-L3572
I agree.  For plugins' authors, this is one more pitfall to be aware of.  For example, you can't write this to replace all occurrences of pat with rep:
:%s/pat/rep/g

Because you have no idea whether the user will have set 'gd' or not.  So, instead, you have to write this:
exe ':%s/pat/rep/' .. (&gd ? 'g' : '')

Notice how much more verbose the command gets.

Besides, I guess the main purpose of this option is to save one (or two) keypress when performing a substitution.  That's hardly relevant when writing a script.

'edcompatible' should be ignored too.
Notice the confusing result a plugin gives if a user has set the option:
vim -es -Nu NONE -S <(cat <<'EOF'
    vim9script
    setline(1, repeat(['aaa xxx aaa xxx'], 4))
    :1s/xxx/bbb/g
    :2s/xxx/bbb/g
    :3s/xxx/bbb/g
    :4s/xxx/bbb/g
    :%p
    qa!
EOF
)

Expected:
aaa bbb aaa bbb
aaa bbb aaa bbb
aaa bbb aaa bbb
aaa bbb aaa bbb

Actual:
aaa bbb aaa bbb
aaa bbb aaa xxx
aaa bbb aaa bbb
aaa bbb aaa xxx


helm install nginx needs chart name

Use Helm to deploy an NGINX ingress controller
helm install nginx stable/nginx-ingress
--namespace ingress-basic
--set controller.replicaCount=2
--set controller.nodeSelector."beta.kubernetes.io/os"=linux
--set defaultBackend.nodeSelector."beta.kubernetes.io/os"=linux
Error: This command needs 1 argument: chart name

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ada58160-dc50-b977-d3bc-65f92406e16d
Version Independent ID: 5c596e04-fddb-effe-3b72-575f4df42d4e
Content: Create ingress with automatic TLS - Azure Kubernetes Service
Content Source: articles/aks/ingress-tls.md
Service: container-service
GitHub Login: @mlearned
Microsoft Alias: mlearned

@microkost  Thanks for your comment. We will review and get back to you shortly.

@microkost I tried running this on Cloud Shell and it is working as expected without passing chart name argument.
Can you please check your cli version. I tried on azure-cli                          2.2.0

az version
This command is in preview. It may be changed/removed in a future release.
{
"azure-cli": "2.3.1",
"azure-cli-command-modules-nspkg": "2.0.3",
"azure-cli-core": "2.3.1",
"azure-cli-nspkg": "3.0.4",
"azure-cli-telemetry": "1.0.4",
"extensions": {
"aks-preview": "0.4.34"
}
}

@microkost Thank you for the feedback.
Are you using Helm 2 or Helm 3? The steps outlined in this article use Helm 3. I have updated doc to make that clearer and the changes should be live soon.


data_engine file cannot be decompressed

Hello, thanks for the great work.
data_engine file cannot be decompressed normally.

Any suggestions about this error.  Thanks.

Be resolved  by running merge_tars.py.


does not work on vue2 with syntax in Readme

This is an odd one because when I first approached you about this vue2 problem you told me the solution was to do
import { RayPlugin } from 'vue-ray/vue2';

Vue.use(RayPlugin.default, {

instead of
import { RayPlugin } from 'vue-ray/vue2';

Vue.use(RayPlugin, {

Which worked at the time, and then you solved it and it was no longer needed.. and for a while I was able to test vue-ray without adding the .default
But today (using latest versions vue-ray 1.16 and node-ray 1.13) the problem started happening again.
It would correctly intercept console.log events (which I opened another issue about because I don't want that), but if I tried to use this.$ray it would throw an error and say that this.$ray is not defined
When I re-added the .default then it started working correctly again

@vesper8 Thank you, I'm looking into this now.

@vesper8 This should be resolved as of version 1.17.1.  The following should work for you with vue 2:
// main.js

Vue.config.productionTip = false

import Vue from 'vue';
import { RayPlugin } from 'vue-ray/vue2';
import App from './App.vue'

Vue.use(RayPlugin, {
  interceptErrors: true,
  host: 'localhost',
  port: 23517,
  showComponentEvents: [
    'created',
    'mounted',
  ],
  nodeRaySettings: {
    interceptConsoleLog: true,
  }
});

new Vue({
  render: h => h(App),
}).$mount('#app');

console.log('loaded');


Update Apache header for the new year

Thanks J, can you re-push this with your work address associated to the commit?

Done


Cherry Pick to 1.1: Configure logging level in proxy and control plane (#12639)

configure proxy log level via helm values for sidecar and gateways

configure istio control plane log level via helm

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: howardjohn
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: linsun
If they are not already assigned, you can assign the PR to them by writing /assign @linsun in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@howardjohn: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
bb4ac6bebedd00a298a4744dd4ecc8194f3796a0
link
/test istio-integ-k8s-tests

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

prow/istio-integ-k8s-tests.sh
bb4ac6bebedd00a298a4744dd4ecc8194f3796a0
link
/test istio-integ-k8s-tests

prow/istio-pilot-multicluster-e2e.sh
bb4ac6bebedd00a298a4744dd4ecc8194f3796a0
link
/test istio-pilot-multicluster-e2e

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/approve
/lgtm

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: GregHanson, howardjohn
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

install/OWNERS [GregHanson]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

cc @hzxuzhonghu could you add the CLA yes label?  myself and John have signed the CLA


Best way to offer "breadcrumbs"

I am currently writing an application that parses config files which optionally are able to include other config files (by chance, they are written in kdl; I am running the zkat stack I guess)
However if an error in one those config files occurs, I want to offer breadcrumbs of where it was included, what would be the best way to achieve this?
I am currently trying to use #[related] but it now looks as if every include is an error lol
  × Found cyclic include for "pneumail-mta/resources/lol.kdl" at pneumail-mta/resources/oof.kdl:0:8 which was entered at pneumail-mta/resources/nyaa.kdl:0:8
   ╭─[pneumail-mta/resources/oof.kdl:1:1]
 1 │ include "lol.kdl"
   ·         ────┬────
   ·             ╰── included again here
   ╰────

Error:   × File was selected by
   ╭─[pneumail-mta/resources/nyaa.kdl:1:1]
 1 │ include "oof.kdl"
   ·         ────┬────
   ·             ╰── included here
   ╰────
Error:   × File was selected by
   ╭─[pneumail-mta/resources/lol.kdl:1:1]
 1 │ try-include "nyaa.kdl"
   ·             ─────┬────
   ·                  ╰── included here
   ╰────
Error:   × File was selected by
   ╭─[pneumail-mta/resources/config.kdl:1:1]
 1 │ include "lol.kdl"
   ·         ────┬────
   ·             ╰── included here
 2 │
   ╰────

Would there be a better or different way to offer these "breadcrumbs"?

I found a really disgusting way to get a bit better looking results, which is making the breadcrumb "errors" start with \r so the whole error prefix gets yeeted, but this works only on consoles lol

You have basically three choices right now:

use the "related" diagnostics feature and stuff your breadcrumbs into a "related" field.
use diagnostic_source, which will "nest" the recursive source diagnostics in the "breadcrumb"
write your own logic for printing multiple diagnostics according to your specific needs.

The difference between 1 and 2 is mostly whether you want things to look nested or not.

Ah okay, yeah 3 is the option that, to me makes most "reader ergonomic" sense, but miette doesn't expose the "annotated source code struct" so I guess the best choice for me is to double dip miette and ariadne. I was just hoping to avoid that, as knuffel already uses miette
anyhow, thanks :)


Reduce queue size and add filter for csr informer

always return single key to queue so it will not blow up the queue
add filter for csr informer

Signed-off-by: Jian Qiu<EMAIL_ADDRESS>

/assign @elgnay

/lgtm


How to custom module label_names

I'm confused how to custom the label_names.
If I comment the line 'model.bind(...)'
the code will raise an error of not find the 'softmax_lable' (as the default name of lable),
But I have pass the label_names param in Module()
With uncomment the line 'model.bind',
the code will work but raise a warnning 'WARNING:root:Already binded, ignoring bind()'
So, what is the right way of set label_names and init the module?
label = mx.sym.Variable('label')
dloss  = mx.symbol.LogisticRegressionOutput(data = fc3, label=label, name = 'dloss')
model =  mx.mod.Module(symbol=dloss,data_names=('data',), label_names=('label',))
#model.bind( data_shapes=train_iter.provide_data,label_shapes=[('label', (batch_size,))])
model.fit(
    train_iter,
    eval_data=test_iter,
    optimizer_params={'learning_rate':0.01, 'momentum': 0.9},
    num_epoch=10)

Please refer the example: https://github.com/dmlc/mxnet/tree/master/example/multi-task to customize data iterator.

Thank you, and as mentioned in #866, after read the code in io.py, the implementation of NDArrayIter. I notice that the label_name can be dictionary...
so if the code changed like:
train_iter = mx.io.NDArrayIter(XY_train[:,:2], {'label':XY_train[:,2]}, batch_size=batch_size)
test_iter = mx.io.NDArrayIter(XY_test[:,:2], {'label':XY_test[:,2]}, batch_size=batch_size)

it can work without call for bind.
I think when I want to set the label, I firstly consider the API from the Module, and a 'label_names' param is indeed here...
Is there some reason that the assignment of label_names in Module not overwrite the value in DataIter?

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


DST-520 - Add more descriptive text for change links when adding mult…

…iple end users

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.23%. Comparing base (e8c5057) to head (37f79ed).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #264   +/-   ##
=======================================
  Coverage   81.23%   81.23%
=======================================
  Files          78       78
  Lines        2356     2356
=======================================
  Hits         1914     1914
  Misses        442      442

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Remove spaces from check taint ( as these can mess with taint checkin…

This fixes #156.
The problem description can be found in the issue.
This pr makes taint checking ignore spaces and changes the multiplication order.
Note that it might be possible that z3 does not swap op0 and op1 in some cases in which case taint checking will fail again. We might want to move to a more robust way of taint checking in the future.

Thanks @JoranHonig! I agree that the taint checking sucks, I wouldn't even say that but qualifies as taint checking. Writing a proper taint engine is certainly on the roadmap.


Add decorators to hotplug tests indicating block or filesystem

What this PR does
Before this PR:
No decorators on hotplug testing to indicate if a test requires block or filesystem volumes
After this PR:
decorators on hotplug testing to indicate if a test requires block or filesystem volumes.
This will allow simpler filtering on specific volume mode tests if the storage provider only supports a particular volume mode.

Fixes #
Why we need it and why it was done in this way
The following tradeoffs were made:
The following alternatives were considered:
Links to places where the discussion took place:
Special notes for your reviewer

Checklist
This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

[ ] Design: A design document was considered and is present (link) or not required
[ ] PR: The PR description is expressive enough and will help future contributors
[ ] Code: Write code that humans can understand and Keep it simple
[X] Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
[ ] Upgrade: Impact of this change on upgrade flows was considered and addressed if required
[ ] Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
[ ] Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
[ ] Community: Announcement to kubevirt-dev was considered

Release note

NONE

/retest


Release 1.7.1 [4458]

This release includes the following features:

LogFileConsumer added to the logging system
Handle FASTRTPS_DEFAULT_PROFILES_FILE environment variable indicating the default profiles XML file
XML parser made more restrictive and with better error messages

It also fixes some important bugs:

Fixed discovery issues related to the selected network interfaces on Windows
Improved discovery times
Workaround ASIO issue with multicast on QNX systems
Improved TCP transport performance
Improved handling of key-only data submessages

Some other minor bugs and performance improvements.

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

Build status:

Linux
Mac
Windows

ROS 2 builds against the default branches (which will become ROS Dashing end of May):

Linux
Linux-aarch64
macOS
Windows


enable-secrets-caching is not documented

Describe the solution you'd like
enable-secrets-caching is a valid extraArg, but it's not documented. It would be great if it were.
What is the added value?
I don't have to bother devs on slack to ask how to enable this.
Give us examples of the outcome
external-secrets devs get to enjoy their day without hearing slack noises.

I've restructured the docs in #1539 (to be merged) and added a section for CLI args here: https://github.com/external-secrets/external-secrets/pull/1539/files#diff-f7b862349bfea07473d1d9f4904410fc6f36309d1d26704fcaa59c347f250430
So it will be there soon :tm:


Feature: Native prometheus metrics

Provide native prometheus /metrics endpoint instead of converting statsd->prometheus metrics which is currently done in the Helm charts for kubernetes
https://prometheus.io/ supported by https://www.cncf.io/

With https://github.com/pinterest/secor/issues/1341, there is integration with micrometer.io which supports prometheus, it should be pretty easy to add prometheus through micrometer.


CMake Error at /usr/local/Cellar/cmake/3.21.0/share/cmake/Modules/CMakeTestCXXCompiler.cmake:59

Describe the bug
A clear and concise description of what the bug is.
when I deploy graphscope on my mac（bigsur 11.1）. using shell below
./scripts/deploy_local.sh install_deps
source ~/.graphscope_env

a cmake error 出现了
Screenshots
If applicable, add screenshots to help explain your problem.
CMake Error at /usr/local/Cellar/cmake/3.21.0/share/cmake/Modules/CMakeTestCXXCompiler.cmake:59 (message):
  The C++ compiler

    "/usr/local/opt/llvm@9/bin/clang++"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /tmp/folly-2<IP_ADDRESS>/_build/CMakeFiles/CMakeTmp

    Run Build Command(s):/usr/bin/make -f Makefile cmTC_f6b71/fast && /Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/cmTC_f6b71.dir/build.make CMakeFiles/cmTC_f6b71.dir/build
    Building CXX object CMakeFiles/cmTC_f6b71.dir/testCXXCompiler.cxx.o
    /usr/local/opt/llvm@9/bin/clang++   -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -mmacosx-version-min=11.1 -fPIE -MD -MT CMakeFiles/cmTC_f6b71.dir/testCXXCompiler.cxx.o -MF CMakeFiles/cmTC_f6b71.dir/testCXXCompiler.cxx.o.d -o CMakeFiles/cmTC_f6b71.dir/testCXXCompiler.cxx.o -c /tmp/folly-2<IP_ADDRESS>/_build/CMakeFiles/CMakeTmp/testCXXCompiler.cxx
    clang-9: error: invalid version number in '-mmacosx-version-min=11.1'
    make[1]: *** [CMakeFiles/cmTC_f6b71.dir/testCXXCompiler.cxx.o] Error 1
    make: *** [cmTC_f6b71/fast] Error 2

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:49 (project)

-- Configuring incomplete, errors occurred!
See also "/tmp/folly-2<IP_ADDRESS>/_build/CMakeFiles/CMakeOutput.log".
See also "/tmp/folly-2<IP_ADDRESS>/_build/CMakeFiles/CMakeError.log".

Environment (please complete the following information):

GraphScope version: [master]
OS: [MacOS]
Version [e.g. 11.1]

Additional context
Add any other context about the problem here.

Hi miziha-zp, it seems that your clang++ is not correctly installed. You may install llvm using brew install llvm and try again. Thanks!

Hi miziha-zp, it seems that your clang++ is not correctly installed. You may install llvm using brew install llvm and try again. Thanks!

thanks, I will have a try

Hi @miziha-zp , hope the issue resolved. If you have further problems, please reopen(open) this(new) issue~


Fan control issue

If the top fan is set to 0, when the goggle starts overheating the top fan goes to 5 but the side fans are not working and the manual control in the fan menu is not working when that happens. The result is the goggle keeps overheating until they are turned off. See video
https://www.youtube.com/watch?v=NaSG6zxUZBI

Top fan minimum speed is now 1 of 5 in test fw

change landed on GH in https://github.com/hd-zero/hdzero-goggle/pull/79


Added GeoBackupPolicy SDK to SQL Azure Database Geo Backup model

Can one of the admins verify this patch?

Hi @alazad-msft, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.azure.com.

TTYL, AZPRBOT;

@alazad-msft, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.
Thanks, AZPRBOT;

@azuresdkci add to whitelist

@alazad-msft please pull from upstream master branch and resolve merge conflicts.

Completed the merge and conflict resolution.


Inconsistency in help info for barcode header in prep template

It is unclear from the help page info whether to use "barcode" or "barcodesequence" as the column header name in a prep template. https://github.com/biocore/qiita/wiki/Preparing-Qiita-template-files lists "barcode" but the downloadable prep template example has a column header called "barcodesequence." The correct header is "barcode", please update the downloadable example template.

Thanks for catching this discrepancy.

Thanks @jessicalmetcalf it looks like the example files are old.


Sending keycodes not working in JS

The following code used to work in JS to send a keycode to an element:
// Backspace
int keyCode = 8;
var e = new KeyEvent('keydown', keyCode: keyCode shiftKey: false);
element.dispatchEvent(e.wrapped);
However, now it gives the following error:
Uncaught Error: InvalidStateError: Failed to execute 'dispatchEvent' on 'EventTarget': The event provided is null.
N.B. This has never worked in Dartium due to this bug:
https://code.google.com/p/dart/issues/detail?id=13902

Why was this closed?


Added new exporters to the TestRunner and reworked the unit tests.

#171

The unit test structure is now entirely different: lots of code was broken and outdated...
The TestRunner project has been updated with multiple exporters.
The TestRunner now doesn't crash on exit.
Fixed MSVC warnings.
Removed the Build folder so as to avoid commit noise.

Note: looks like everything works just fine with VS2017!

This PR also adds unit test enabling control via the Projucer (and juce+cmake I'm guessing).

Thanks @jrlanglois! Much of this looks really great. As I'm reading the changes, I wonder, would you be interested in driving this all the way as in https://github.com/nick-thompson/blueprint/issues/136? Generally I think what's laid out in that issue will be much more valuable than the unit tests we have here

As I'm reading the changes, I wonder, would you be interested in driving this all the way as in #136? Generally I think what's laid out in that issue will be much more valuable than the unit tests we have here

Let's do it!


ImportError: cannot import name 'YouTube'

the package was installed.
can import youtube
can from youtube import YouTubeException
cannot from youtube import YouTube

ImportError: cannot import name 'YouTube'

what should i do?

I'm sorry for the wrong instruction, I have updated the README.md for better clarity and you can try this
from youtube import API
api = API(api_key='')

This should work.


Experienced '504 Deadline Exceeded' error when tried to delete large number rows by partitioned dml

There are 1 billion old records need to be cleaned. According the documents, It seems Partitioned DML should be the good choice. I choose java client because we can set timeout by using setPartitionedDmlTimeout(Duration.ofHours(24L)). However, I still experienced the '504 Deadline Exceeded' after a few hours that shorter than 24 hours I set.  Why setPartitionedDmlTimeout did not work? How should I solve the problem? Is it not possible to delete so large number of data by using Partitioned DML.
Following are the details.
Environment details

Spanner
3 nodes, 2 billion records in a table and 1 billion old records need to be cleaned
OS type and version: MacOs version 10.15.3
Java version: java 8
spanner version(s): the latest version

Steps to reproduce

Write 2 billion records with timestamp in to the spanner table with dataflow
Run a clean job to delete  the old 1 billion records with Partitioned DML by java client
Set the timeout to be 24 hours by using setPartitionedDmlTimeout(Duration.ofHours(24L))
Run the java code

Code example
SpannerOptions options = SpannerOptions.newBuilder().setPartitionedDmlTimeout(Duration.ofHours(24L)).build();

        Spanner spanner = options.getService();
        try {

            DatabaseId db = DatabaseId.of(options.getProjectId(), args[0], args[1]);
            // [END init_client]
            //This will return the default project id based on the environment.
            String clientProject = spanner.getOptions().getProjectId();
            if (!db.getInstanceId().getProject().equals(clientProject)) {
                System.err.println(
                        "Invalid project specified. Project in the database id should match the"
                                + "project name set in the environment variable GOOGLE_CLOUD_PROJECT. Expected: "
                                + clientProject);
            }

            // [START init_client]
            DatabaseClient dbClient = spanner.getDatabaseClient(db);

            SimpleDateFormat formatter = new SimpleDateFormat("dd/MM/yyyy hh:mm:ss.SSS");
            String sql="DELETE FROM MyTable WHERE updateTimestamp<'2020-05-02T03:00:00Z'";
            Date currentDate = new Date();
            System.out.println(String.format("Delete Start:%s",formatter.format(currentDate)));

            long lStartTime = Instant.now().toEpochMilli();
            deleteUsingPartitionedDml(dbClient,sql);
            long lEndTime = Instant.now().toEpochMilli();
            long elaspedTime = lEndTime - lStartTime;
            System.out.println("Elapsed time in : " + elaspedTime/1000);

        } finally {
            spanner.close();
        }
        // [END init_client]
        System.out.println("Closed client");
    }
    private static void deleteUsingPartitionedDml(DatabaseClient dbClient,String sql) {
        long rowCount = dbClient.executePartitionedUpdate(Statement.of(sql));

        System.out.printf("%d records deleted.\n", rowCount);
    }

@FengnaLiu Thanks for the detailed report. I'll look into it.
Could you also share the stacktrace of the exception that you get? The reason I'm asking is that I noticed that you wrote that you get a 504 Deadline Exceeded error, which indicates that it is an HTTP error and not a gRPC error. That could indicate that the timeout is occurring on the server and not in the client. The client would normally throw a DEADLINE_EXCEEDED exception with gRPC status code 4.

@FengnaLiu I have a couple of additional questions in addition to the one above.

Could it be that the Partitioned DML statement is at least partly executed? In other words: If you execute a SELECT COUNT(*) FROM MyTable before and after running the above sample, does it return the same value?
Does your network infrastructure include a HTTP proxy?

@olavloite I tried to run the delete job again. I got the  'DEADLINE_EXCEEDED' instead of '504 DEADLINE_EXCEEDED'  about four hours later after started the delete job.
Following are the details.

Half of the data was deleted.

The stacktrace of the exception i got
com.google.cloud.spanner.SpannerException: DEADLINE_EXCEEDED: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after<PHONE_NUMBER>514ns. [buffered_nanos=186035342, remote_addr=spanner.googleapis.com/<IP_ADDRESS>:443]  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted (SpannerExceptionFactory.java:182)  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:45)  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:139)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:691)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:543)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:87)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:69)  at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)  at com.google.cloud.RetryHelper.run (RetryHelper.java:76)  at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)  at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:47)  at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:91)  at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:103)  at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:647)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:199)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:196)  at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:213)  at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:194)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:93)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:81)  at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke (Method.java:498)  at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)  at java.lang.Thread.run (Thread.java:748) Caused by: java.util.concurrent.ExecutionException: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after<PHONE_NUMBER>514ns. [buffered_nanos=186035342, remote_addr=spanner.googleapis.com/<IP_ADDRESS>:443]  at com.google.common.util.concurrent.AbstractFuture.getDoneValue (AbstractFuture.java:552)  at com.google.common.util.concurrent.AbstractFuture.get (AbstractFuture.java:533)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:685)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:543)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:87)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:69)  at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)  at com.google.cloud.RetryHelper.run (RetryHelper.java:76)  at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)  at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:47)  at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:91)  at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:103)  at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:647)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:199)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:196)  at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:213)  at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:194)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:93)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:81)  at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke (Method.java:498)  at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)  at java.lang.Thread.run (Thread.java:748) Caused by: com.google.api.gax.rpc.DeadlineExceededException: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after<PHONE_NUMBER>514ns. [buffered_nanos=186035342, remote_addr=spanner.googleapis.com/<IP_ADDRESS>:443]  at com.google.api.gax.rpc.ApiExceptionFactory.createException (ApiExceptionFactory.java:51)  at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:72)  at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:60)  at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure (GrpcExceptionCallable.java:97)  at com.google.api.core.ApiFutures$1.onFailure (ApiFutures.java:68)  at com.google.common.util.concurrent.Futures$CallbackListener.run (Futures.java:982)  at com.google.common.util.concurrent.DirectExecutor.execute (DirectExecutor.java:30)  at com.google.common.util.concurrent.AbstractFuture.executeListener (AbstractFuture.java:1138)  at com.google.common.util.concurrent.AbstractFuture.complete (AbstractFuture.java:957)  at com.google.common.util.concurrent.AbstractFuture.setException (AbstractFuture.java:748)  at io.grpc.stub.ClientCalls$GrpcFuture.setException (ClientCalls.java:520)  at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:495)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose (CensusStatsModule.java:700)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose (CensusTracingModule.java:399)  at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:510)  at io.grpc.internal.ClientCallImpl.access$300 (ClientCallImpl.java:66)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:630)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700 (ClientCallImpl.java:518)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:692)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:681)  at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)  at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)  at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)  at java.util.concurrent.FutureTask.run (FutureTask.java:266)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)  at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)  at java.lang.Thread.run (Thread.java:748) Caused by: io.grpc.StatusRuntimeException: DEADLINE_EXCEEDED: deadline exceeded after<PHONE_NUMBER>514ns. [buffered_nanos=186035342, remote_addr=spanner.googleapis.com/<IP_ADDRESS>:443]  at io.grpc.Status.asRuntimeException (Status.java:533)  at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:495)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose (CensusStatsModule.java:700)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose (CensusTracingModule.java:399)  at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:510)  at io.grpc.internal.ClientCallImpl.access$300 (ClientCallImpl.java:66)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:630)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700 (ClientCallImpl.java:518)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:692)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:681)  at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)  at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)  at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)  at java.util.concurrent.FutureTask.run (FutureTask.java:266)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)  at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)  at java.lang.Thread.run (Thread.java:748)

My network infrastructure does not include a HTTP proxy

@FengnaLiu Thanks for the additional information. I have been able to reproduce this, and a fix has been merged. You could try it with that by checking out the latest version of the master branch of this repository, or wait for the next release.

@olavloite Any idea when the next release will be?

@iamacleish A new release has now been published.

@olavloite Thanks for your information.
I tried the new version. But I encountered the  new error [com.google.cloud.spanner.SpannerException: INTERNAL: ] after 1 hour running.
Details information as following.

The stacktrace of the exception i got
com.google.cloud.spanner.SpannerException: INTERNAL: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Internal error encountered.  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted (SpannerExceptionFactory.java:210)  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:56)  at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:150)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:1214)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:1060)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:86)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:68)  at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)  at com.google.cloud.RetryHelper.run (RetryHelper.java:76)  at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)  at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:57)  at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:90)  at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:108)  at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:688)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:205)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:202)  at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:219)  at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:200)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:56)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:44)  at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke (Method.java:498)  at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)  at java.lang.Thread.run (Thread.java:748) Caused by: java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Internal error encountered.  at com.google.common.util.concurrent.AbstractFuture.getDoneValue (AbstractFuture.java:564)  at com.google.common.util.concurrent.AbstractFuture.get (AbstractFuture.java:545)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:1208)  at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:1060)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:86)  at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:68)  at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)  at com.google.cloud.RetryHelper.run (RetryHelper.java:76)  at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)  at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:57)  at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:90)  at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:108)  at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:688)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:205)  at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:202)  at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:219)  at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:200)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:56)  at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:44)  at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)  at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)  at java.lang.reflect.Method.invoke (Method.java:498)  at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)  at java.lang.Thread.run (Thread.java:748) Caused by: com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: Internal error encountered.  at com.google.api.gax.rpc.ApiExceptionFactory.createException (ApiExceptionFactory.java:67)  at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:72)  at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:60)  at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure (GrpcExceptionCallable.java:97)  at com.google.api.core.ApiFutures$1.onFailure (ApiFutures.java:68)  at com.google.common.util.concurrent.Futures$CallbackListener.run (Futures.java:1050)  at com.google.common.util.concurrent.DirectExecutor.execute (DirectExecutor.java:30)  at com.google.common.util.concurrent.AbstractFuture.executeListener (AbstractFuture.java:1176)  at com.google.common.util.concurrent.AbstractFuture.complete (AbstractFuture.java:969)  at com.google.common.util.concurrent.AbstractFuture.setException (AbstractFuture.java:760)  at io.grpc.stub.ClientCalls$GrpcFuture.setException (ClientCalls.java:526)  at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:501)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)  at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:426)  at io.grpc.internal.ClientCallImpl.access$500 (ClientCallImpl.java:66)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:689)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900 (ClientCallImpl.java:577)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:751)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:740)  at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)  at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)  at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)  at java.util.concurrent.FutureTask.run (FutureTask.java:266)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)  at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)  at java.lang.Thread.run (Thread.java:748) Caused by: io.grpc.StatusRuntimeException: INTERNAL: Internal error encountered.  at io.grpc.Status.asRuntimeException (Status.java:533)  at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:501)  at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)  at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)  at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)  at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)  at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:426)  at io.grpc.internal.ClientCallImpl.access$500 (ClientCallImpl.java:66)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:689)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900 (ClientCallImpl.java:577)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:751)  at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:740)  at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)  at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)  at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)  at java.util.concurrent.FutureTask.run (FutureTask.java:266)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)  at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)  at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)  at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)  at java.lang.Thread.run (Thread.java:748)

pom.xml
`

com.google.cloud
libraries-bom
5.4.0
pom
import

UTF-8
UTF-8

<!--testing dependencies-->
<dependency>
  <groupId>junit</groupId>
  <artifactId>junit</artifactId>
  <version>4.13</version>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>org.hamcrest</groupId>
  <artifactId>hamcrest-library</artifactId>
  <version>2.2</version>
  <scope>test</scope>
</dependency>

<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-databind</artifactId>
  <version>2.11.0</version>
</dependency>

<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-spanner</artifactId>
  <version>1.55.0</version>
</dependency>

`

@FengnaLiu That's surprising. I'll look into it right away. Did you set any specific timeout value?

@olavloite   I did set the timeout to be 24 hours.
SpannerOptions options = SpannerOptions.newBuilder().setPartitionedDmlTimeout(Duration.ofHours(24L)).build();

@olavloite I ran the job again and then the error seems different.
com.google.cloud.spanner.AbortedException: ABORTED: com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Aborted due to transient fault     at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerExceptionPreformatted (SpannerExceptionFactory.java:196)     at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:56)     at com.google.cloud.spanner.SpannerExceptionFactory.newSpannerException (SpannerExceptionFactory.java:150)     at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:1214)     at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:1060)     at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:86)     at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:68)     at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)     at com.google.cloud.RetryHelper.run (RetryHelper.java:76)     at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)     at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:57)     at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:90)     at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:108)     at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:688)     at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:205)     at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:202)     at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:219)     at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:200)     at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:56)     at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:44)     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)     at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke (Method.java:498)     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)     at java.lang.Thread.run (Thread.java:748) Caused by: java.util.concurrent.ExecutionException: com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Aborted due to transient fault     at com.google.common.util.concurrent.AbstractFuture.getDoneValue (AbstractFuture.java:564)     at com.google.common.util.concurrent.AbstractFuture.get (AbstractFuture.java:545)     at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.get (GapicSpannerRpc.java:1208)     at com.google.cloud.spanner.spi.v1.GapicSpannerRpc.executePartitionedDml (GapicSpannerRpc.java:1060)     at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:86)     at com.google.cloud.spanner.PartitionedDMLTransaction$1.call (PartitionedDMLTransaction.java:68)     at com.google.api.gax.retrying.DirectRetryingExecutor.submit (DirectRetryingExecutor.java:105)     at com.google.cloud.RetryHelper.run (RetryHelper.java:76)     at com.google.cloud.RetryHelper.runWithRetries (RetryHelper.java:50)     at com.google.cloud.spanner.SpannerRetryHelper.runTxWithRetriesOnAborted (SpannerRetryHelper.java:57)     at com.google.cloud.spanner.PartitionedDMLTransaction.executePartitionedUpdate (PartitionedDMLTransaction.java:90)     at com.google.cloud.spanner.SessionImpl.executePartitionedUpdate (SessionImpl.java:108)     at com.google.cloud.spanner.SessionPool$PooledSession.executePartitionedUpdate (SessionPool.java:688)     at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:205)     at com.google.cloud.spanner.DatabaseClientImpl$3.apply (DatabaseClientImpl.java:202)     at com.google.cloud.spanner.DatabaseClientImpl.runWithSessionRetry (DatabaseClientImpl.java:219)     at com.google.cloud.spanner.DatabaseClientImpl.executePartitionedUpdate (DatabaseClientImpl.java:200)     at jp.co.sej.central.spanner.DeleteOldStoreRecords.deleteUsingPartitionedDml (DeleteOldStoreRecords.java:56)     at jp.co.sej.central.spanner.DeleteOldStoreRecords.main (DeleteOldStoreRecords.java:44)     at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)     at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)     at java.lang.reflect.Method.invoke (Method.java:498)     at org.codehaus.mojo.exec.ExecJavaMojo$1.run (ExecJavaMojo.java:282)     at java.lang.Thread.run (Thread.java:748) Caused by: com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Aborted due to transient fault     at com.google.api.gax.rpc.ApiExceptionFactory.createException (ApiExceptionFactory.java:61)     at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:72)     at com.google.api.gax.grpc.GrpcApiExceptionFactory.create (GrpcApiExceptionFactory.java:60)     at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure (GrpcExceptionCallable.java:97)     at com.google.api.core.ApiFutures$1.onFailure (ApiFutures.java:68)     at com.google.common.util.concurrent.Futures$CallbackListener.run (Futures.java:1050)     at com.google.common.util.concurrent.DirectExecutor.execute (DirectExecutor.java:30)     at com.google.common.util.concurrent.AbstractFuture.executeListener (AbstractFuture.java:1176)     at com.google.common.util.concurrent.AbstractFuture.complete (AbstractFuture.java:969)     at com.google.common.util.concurrent.AbstractFuture.setException (AbstractFuture.java:760)     at io.grpc.stub.ClientCalls$GrpcFuture.setException (ClientCalls.java:526)     at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:501)     at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)     at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)     at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)     at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)     at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:426)     at io.grpc.internal.ClientCallImpl.access$500 (ClientCallImpl.java:66)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:689)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900 (ClientCallImpl.java:577)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:751)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:740)     at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)     at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)     at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)     at java.util.concurrent.FutureTask.run (FutureTask.java:266)     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)     at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)     at java.lang.Thread.run (Thread.java:748) Caused by: io.grpc.StatusRuntimeException: ABORTED: Aborted due to transient fault     at io.grpc.Status.asRuntimeException (Status.java:533)     at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose (ClientCalls.java:501)     at io.grpc.PartialForwardingClientCallListener.onClose (PartialForwardingClientCallListener.java:39)     at io.grpc.ForwardingClientCallListener.onClose (ForwardingClientCallListener.java:23)     at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose (ForwardingClientCallListener.java:40)     at com.google.cloud.spanner.spi.v1.SpannerErrorInterceptor$1$1.onClose (SpannerErrorInterceptor.java:100)     at io.grpc.internal.ClientCallImpl.closeObserver (ClientCallImpl.java:426)     at io.grpc.internal.ClientCallImpl.access$500 (ClientCallImpl.java:66)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close (ClientCallImpl.java:689)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900 (ClientCallImpl.java:577)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal (ClientCallImpl.java:751)     at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext (ClientCallImpl.java:740)     at io.grpc.internal.ContextRunnable.run (ContextRunnable.java:37)     at io.grpc.internal.SerializingExecutor.run (SerializingExecutor.java:123)     at java.util.concurrent.Executors$RunnableAdapter.call (Executors.java:511)     at java.util.concurrent.FutureTask.run (FutureTask.java:266)     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201 (ScheduledThreadPoolExecutor.java:180)     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run (ScheduledThreadPoolExecutor.java:293)     at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)     at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)     at java.lang.Thread.run (Thread.java:748)

@FengnaLiu Thanks for the additional information. That is indeed a different error. The Partitioned DML transaction has been aborted, and normally that should lead to a retry by the client, but it seems that that is not happening in this case. I'll try to figure out why that is.

@FengnaLiu The last problem you ran into seems to be caused by the following scenario:

The Partitioned DML transaction and statement is started.
The transaction is aborted by Cloud Spanner.
The client starts an automatic retry of the PDML transaction. The retry and timeout settings that were used by the automatic retryer for the retried transaction were the default retry settings, and not the specific PDML retry settings that have been introduced.

The last point above has now also been changed.

@olavloite I really appreciate your quick response.
There are still something that I am not clear.

Does it mean I have to wait for next release to apply the changes?
And how about the first problem [com.google.cloud.spanner.SpannerException: INTERNAL: ]?
Did it also have been fixed?

@olavloite I really sorry to bother you again. I tried version 1.55.1 and 1.55.0 for several delete patterns. And I found out that with version 1.55.1 it did experience no timeout error but the job kept running even the records have been all deleted. Additionally, it appears to deleting much far more slower than the older version.
Details as followings:

delete 1 billion records.
Version 1.55.0 - The job experiences timeout error after a few hours running , and timeout I set is 24 hours
Version 1.55.1  - the job keeps running even after 24 hours( the timeout I set). Additionally, I checked the left records during the job and it appears to stop deleting records after 5-6 hours.

delete 100million records.
Version 1.55.0 and Version 1.46.0 - Job Succeeded after 30 minutes.
Version 1.55.1 -  After 2 hours the job is still running and I checked the left record number through query is 0.  I manually stopped the job after 3 hours.

delete 200 million records
Version 1.55.1 - Job succeeded after 6 hours. I was not able to check when the records were all deleted because I ran the job during the night.
Version 1.55.0 - Not checked yet. I will find out today.

Hi @FengnaLiu

I really sorry to bother you again.
Don't worry about that. This is valuable information for improving the client library.

Thanks for the detailed information and examples. I don't think the new version is literally deleting the records slower, but it might be that it is slower and less reliable in detecting when it has finished.
The big challenge here is that the Partitioned DML RPC is defined as an RPC that returns its response when the operation has finished. That works well for operations that normally takes seconds or maybe a couple of minutes, but not so well if the operation can take hours as any network hick-up during this time might cause unexpected errors. That is why for example RPCs to update the DDL of a database respond with a long-running operation that can be polled for its progress, instead of waiting to respond until the operation has finished on the backend.
I'll start a discussion about this with a couple of others to see if we can come up with a better solution for this. One solution might be to create a kind of long-running operation in the client library that handles all this for you. In the meantime, I'm afraid that your best option is to retry the delete operation with reasonable chunks of the table as you are already doing.
I'll get back to you on this as soon as I can.

Hey @olavloite !
Any updates on this?

@olavloite and I spoke yesterday about whether we can update the client library to support this case. I have a couple of issues with this:

Does it make sense for us to retry these transactions on the client library? There may be some queries where this doesn't make sense?
Under what errors should we retry?

I'm a bit concerned about making the client library logic more complicated than it already is. It also means that we'll have an inconsistency between how Java handles this vs client libraries for other languages.
Ideally the backend would return a LRO here instead of a standard response which would make it easier for us to consistently handle this type of issue across all the client libraries. But I don't see a change like this being made anytime soon.


fix: report failure when at least one of the target is failing

fix: report failure when at least one of the target is failing
Closes #632

Test
To test this pull request, you can run the following commands:
cp <to_package_directory>
go test

Additional Information
Tradeoff

Potential improvement

LGTM but I defer to @olblak because it might break some use cases that we did not htink about.

If I remember correctly, the motivation to fail early after one failing target was to save some time (api call) but I think your suggestion improves the user experience as we see immediately what target is succeeding and those failing.

Correct ! Thanks for the memory. If I'm not mistaken, it was beforr you reworked the scm to have pullrequest/scm as top level: the proposal here makes sense now (while it was hard to implement before the rework).

LGTM (under the assumption that manual testing has to be done)

I'll do some test at the end of the day

@lemeurherve I am opening a pullrequest on this one with a suggestion

Oops sorry I pushed directly to your branch :/, feel free to review my commit

With the manifest that I mention earlier, I get the following output
+++++++++++
+ PREPARE +
+++++++++++

Loading Pipeline "/tmp/test.yaml"

SCM repository retrieved: 0

++++++++++++
+ PIPELINE +
++++++++++++

#############
# TEST.YAML #
#############

TARGETS
========

4
-

**Dry Run enabled**

The shell 🐚 command "false" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----

3
-

**Dry Run enabled**

The shell 🐚 command "false" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----

2
-

**Dry Run enabled**

The shell 🐚 command "false" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----

1
-

**Dry Run enabled**

The shell 🐚 command "true" ran successfully with the following output:
----
----
No change detected

ERROR: something went wrong in target "4" : "Shell command exited on error."
ERROR: something went wrong in target "3" : "Shell command exited on error."
ERROR: something went wrong in target "2" : "Shell command exited on error."

Pipeline "TEST.YAML" failed
Skipping due to:
	"targets stage:\t\"something went wrong during target execution\""
targets stage:	"something went wrong during target execution"

=============================

REPORTS:

✗ TEST.YAML:
	Sources:
	Target:
		✔ [1]  (kind: shell)
		✗ [2]  (kind: shell)
		✗ [3]  (kind: shell)
		✗ [4]  (kind: shell)

Run Summary
===========
Pipeline(s) run:
  * Changed:	0
  * Failed:	1
  * Skipped:	0
  * Succeeded:	0
  * Total:	1
ERROR: ✗ 1 over 1 pipeline failed
ERROR: command failed

@olblak LGTM 😃

Please note that we don't handle the case where a target depends on another target which already failed

Please note that we don't handle the case where a target depends on another target which already failed
I am wondering if it's something we want like for source

I am planning to implement the missing depends on ccheck

title: Test Various target scenario

targets:
  1:
    name: Should be succeeding
    kind: shell
    spec:
      command: "true"

  2:
    name: Should be failling
    kind: shell
    spec:
      command: "false"

  3:
    name: Should be failling
    kind: shell
    spec:
      command: "false"

  4:
    name: Should be skipped
    depends_on:
      - 2
    kind: shell
    spec:
      command: "false"

  5:
    name: Should be succeeding
    kind: shell
    depends_on:
      - 1
    spec:
      command: "true"

output
+++++++++++
+ PREPARE +
+++++++++++

Loading Pipeline "e2e/updatecli.d/command.yaml"

SCM repository retrieved: 0

++++++++++++
+ PIPELINE +
++++++++++++

################################
# TEST VARIOUS TARGET SCENARIO #
################################

TARGETS
========

2
-

**Dry Run enabled**

The shell 🐚 command "false" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----

1
-

**Dry Run enabled**

The shell 🐚 command "true" ran successfully with the following output:
----
----
No change detected

4
-
WARNING: Parent target["2"] did not succeed. Skipping execution of the target["4"]

3
-

**Dry Run enabled**

The shell 🐚 command "false" exited on error (exit code 1) with the following output:
----
----

command stderr output was:
----
----

5
-

**Dry Run enabled**

The shell 🐚 command "true" ran successfully with the following output:
----
----
No change detected

ERROR: something went wrong in target "2" : "Shell command exited on error."
ERROR: something went wrong in target "3" : "Shell command exited on error."

Pipeline "Test Various target scenario" failed
Skipping due to:
	targets stage:	"something went wrong during target execution"

=============================

REPORTS:

✗ COMMAND.YAML:
	Sources:
	Target:
		✔ [1] Should be succeeding (kind: shell)
		✗ [2] Should be failling (kind: shell)
		✗ [3] Should be failling (kind: shell)
		- [4] Should be skipped (kind: shell)
		✔ [5] Should be succeeding (kind: shell)

Run Summary
===========
Pipeline(s) run:
  * Changed:	0
  * Failed:	1
  * Skipped:	0
  * Succeeded:	0
  * Total:	1
ERROR: ✗ 1 over 1 pipeline failed
ERROR: command failed
[1]    18856 exit 1     ./bin/updatecli diff --config e2e/updatecli.d/command.yaml

I am waiting for a final approval before merging this PR

After some test, I didn't catch any regression

Thanks @lemeurherve


JMeter didn't see plugin

Jmeter version: 5.5
Java: openjdk <IP_ADDRESS> 2022-08-18 OpenJDK Runtime Environment (build <IP_ADDRESS>+1-1) OpenJDK 64-Bit Server VM (build <IP_ADDRESS>+1-1, mixed mode, sharing)
OS: Windows Windows 10 Pro 22H2
Plugin: jmeter-plugins-influxdb2-listener-2.6
Error message in GUI JMeter log:

Error setting class: 'io.github.mderevyankoaqa.influxdb2.visualizer.InfluxDatabaseBackendListenerClient' in BackendListener: Backend Listener, check for a missing jar in your jmeter 'search_paths' and 'plugin_dependency_paths' properties>

I just trying to start jmx file from GUI and get this error :(

omg i found reason of this problem, it cuz i started JMeter from ApacheJMeter.jar but not jmeter.bat facepalm


Limits for submodules detection

It would be great a option to disable the 'recursive' submodules detection, or setting the depth level of this detection.
The motivation for this is that opening big projects with many levels of submodules, the git feature from vs become slow and polluted

I'm closing this issue because the recent feature of the "Repository detection" solve my problem, disabling the submodules detection and limiting the repository detection to only opened files.
{
    "git.detectSubmodules": false,
    "git.repositoryScanMaxDepth": 1,
    "git.autoRepositoryDetection": "openEditors"
}


added release automation

Automate the release process to speed things up a bit :)

@tobrun would love to get this in before our next release so I can sure everything works properly.


Swap confident check

Make it toggleable to set what the confident check does

set not confident
or
set confident as default

Added to main


OSD-4181: Unify PHONY targets

There were extra hyphens.

/lgtm

Thanks @lisa ! looks good to me, squash first and I'll merge.

Thanks @lisa ! looks good to me, squash first and I'll merge.

Squashed.

/approve
/lgtm

/retest


determine the registry cred wrt image

RHTAP-3327

I think we might have better solution - your solution kind of expects users to update their jenkinsfiles by adding the ARTIFACTORY_IO_CREDS. These creds are there not by default. I it would be better "just" to rename the default creds name we expect (https://github.com/redhat-appstudio/tssc-sample-templates/blob/main/skeleton/ci/source-repo/jenkins/Jenkinsfile#L15) to something "generic" like IMAGE_REGISTRY_CREDS as there are no real differences between quay/nexus/artifactory if I'm not missing anything. If we rename this, we wouldn't need logic from this PR imo.

In github we chose these names
setVars IMAGE_REGISTRY quay.io/$QUAY_IO_CREDS_USR
setVars IMAGE_REGISTRY_USER $QUAY_IO_CREDS_USR
setVars IMAGE_REGISTRY_PASSWORD $QUAY_IO_CREDS_PSW
so I think the fix is

check if these IMAGE_REGISTRY_USER  IMAGE_REGISTRY_PASSWORD  have values  and if so use them
if no value, use your patch to initialize these values from ARTIFACTORY or QUAY versions

I have made the changes and incorporated your pull request changes. Thanks @prietyc123


bug: use ID of the project created by the benchmarking script

Currently the project ID isn't specified and defaults to whatever is set in the system, making this nondeterministic/flakey.

@googleapis/yoshi-java can anyone review this PR? Thanks!


fix some reference links to point at respective homepage

This is a simple README fix for reference links since they didn't seem to be pointing to the right place.

Nice! Thanks @longcao!

Current coverage is 93.35%

Merging #426 into master will decrease coverage by -0.71% as of 7f1e103

Powered by Codecov. Updated on successful CI builds.


Update B2C default quota limits (total number of objects)

Updating to correct numbers for the default object limit for B2C tenants. The default limit should be 1 million which rises to 5 million on the addition of a verified domain.

@kengaderdus

Can you review this PR?
IMPORTANT: When this content is ready to merge, you must add #sign-off in a comment or the approval may get overlooked.

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team


model: Phi-3 Mini (update)

Motivation
Make sure the Hub is using the latest and most accurate model provided by Microsoft to maintain optimal performance and compatibility.
Specs

Update the model to Microsoft Phi3's latest version.
Verify that the new model works seamlessly with existing infrastructure.
Update documentation to reflect the changes.

Designs
Figma
Tasklist

[ ]

Not in Scope

Changes to other models or components not directly related to Microsoft Phi3.

Appendix

Microsoft Phi3 update release notes:
https://huggingface.co/microsoft/Phi-3-mini-4k-instruct/tree/main
https://huggingface.co/microsoft/Phi-3-mini-128k-instruct

This model is outdated. Will update phi 3.5 later


"Water" isn't a visable inventory item and can seriously weigh you down.

Describe the bug
It is possible to craft "Water" at a cooking pot,  but there is no way to remove it as it does not show up in the inventory.
To Reproduce
Steps to reproduce the behavior:

Go to 'a cooking pot, I was specifically using the one at the Riverwood trader'
Click on 'Craft Water a whole bunch'
Open your Inventory and see that your carry weight has increased
It is now only possible to remove the "Water" from your inventory by crafting with it

Expected behavior
I would like to be able to remove the water from my pockets.
Desktop (please complete the following information):

OS version: Windows 11
RAM: 16gb
CPU: AMD Ryzen 5 3600
GPU: RTX 3080
Storage type Yggdrasil VR is installed on: NVME SSD
Antivirus Windows Defender

please post your save as well

Save77_8C65BE63_0_5661726772_RiverwoodRiverwoodTrader_000507_20220722224223_4_1.zip

Fixed, "Water" should show in your inventory next hotfix.


vcpkg does not work with cmake at all!

When I have installed a package with vcpkg, I cannot use it in cmake, and I have already specified the toolchain file.

How do you set the toolchain file?

I make this one, it worked well. I also build docker image, but the image size over 5GB.
Hope this sample help you
https://github.com/badpaybad/cmakes-helloworld

Hi @zgxnet
Thanks for posting this issue.
Could you please help provide the steps to reproduce your problem?
Which package did you install? What problem did you encounter on which platform? If possible, please also provide your project sample.


fixes #699 by replacing .trim_right_matches with .trim_end_matches

fixes #699 by replacing .trim_right_matches with .trim_end_matches.

Coverage increased (+0.03%) to 40.489% when pulling 9ebc09bd76b6daf31ceb61c6dc9fdead039dc687 on lutzhorn:fix-rust-warnings into 9d8fb5349e3a4bf686b69b7d20f04f5fd122d74b on newsboat:master.

Coverage increased (+0.03%) to 40.489% when pulling 9ebc09bd76b6daf31ceb61c6dc9fdead039dc687 on lutzhorn:fix-rust-warnings into 9d8fb5349e3a4bf686b69b7d20f04f5fd122d74b on newsboat:master.

The changes made by this PR require a current Rust version. The failing CI steps above use Rust 1.26.0, not the current version rustc 1.39.0 (4560ea788 2019-11-04). I am not sure which Rust version should be required to build newsboat.

Hi! Thank you for the pull request. The 1.26 is indeed the lowest supported version; we mention that in the README.
I will start bumping the lowest version and fixing those warnings once I took a look at other Rust apps and figured out what policies they use. I want Newsboat to be buildable by as many people as possible, but also don't want to expend too much effort fighting old warts and compiler bugs.
I'm going to close this, but please don't feel discouraged from submitting more. Take a look at good first issue, help wanted and rust-port tags — maybe something will strike your fancy. Cheers!


Rich mode containers: Are they safe to run?

Question

Rich mode containers: Are they safe to run?
In docs it says:

Also --privileged is required in this situation.

In docker environment --privileged container isn't considered as safe to run as root. So is it same in pouch containers? Or is it safe somehow? Or runv is the way to go for untrusted workloads?
Rich mode container is exactly what I was looking for. Please consider adding more realistic scenario in documentation instead of sleep 10000? such as running webserver and database on it.
Thank you.

Hi @shinebayar-g, thanks for asking. The rich mode is used to setup systemd/initd/supervisor as pid 1 process. Basically, it is used to be reaper to avoid zombie processes and provide systemctl functionality. And sleep 10000 is just as example to show how to using it. You can change it to your own process.

Indeed, that's exactly what I needed. However that doesn't answer my question about --privileged flag concern.

Indeed, that's exactly what I needed. However that doesn't answer my question about --privileged flag concern.

Yeah. It might have security issue. It depends on your case. The workload is trust and it is fine. Might be rootless can help you if the workload is untrusted.

Kata Containers is your choice


Can't connect to a container using machine private IP or host.docker.internal

Version
0.6.0
What steps are needed to reproduce the behavior?

Start a container with a port exposed.
Connect to it using "localhost" – works.
Connect using "host.docker.internal" or host machine private IP – doesn't work. Connection refused.

What you expected?
Connecting successfully.
What happened?
Connection refused.
Also, may be related or not: after installing Stevedore, Docker Desktop no longer has bridge network plugin installed, so it fails to create any networks. I don't run Stevedore and Docker Desktop at the same time. After uninstalling Stevedore, the plugin magically reappears and DD starts working properly again.
Additional context
No response

First, WRT host.docker.internal: I believe it has a different purpose. It is used to connect from container to host. While Stevedore doesn't automagically write /etc/hosts entry for it, you can use --add-host=host.docker.internal:host-gateway to add it. See https://github.com/moby/moby/pull/40007, this is a standard Docker feature.
As for connecting to container using host IP - I need to do some investigation, no comments here yet.

yeah, I know. I use host.docker.internal in my connection strings to postgres, elastic, etc., because I often launch my app as a container in the same network, but sometimes I launch it outside of docker. When I launch it alongside other containers I should use service names instead of IP addresses, and when I launch it as a standalone app I should use localhost. But I don't like having multiple config files for local dev or change values all the time, so I use host.docker.internal instead. It works for both cases, because it's just an alias for private IP.
And while Stevedore doesn't write to hosts, DD does, so I already have host.docker.internal resolving to the host's private IP. When it fails to connect there is an error saying that connection to my private IP (not host.docker.internal) refused. So it resolves just fine.

Perhaps this is related to how WSL works? It does not expose the ports on your LAN adapter by default, however, you can make explicit mappings:
https://docs.microsoft.com/en-us/windows/wsl/networking#accessing-a-wsl-2-distribution-from-your-local-area-network-lan

Stevedore is no longer providing support for Linux containers. See why. I recommend switching to Rancher Desktop, Container Desktop or Colima.


Use contract public key for entire contract state machine

This PR completes earlier work I did to change net functions to use public keys when they are known. Public keys for SendOrderFulfillment, SendOrderCompletion, SendDisputeOpen, SendDisputeClose can now be passed in; in all cases, the places where where they are called have access to contract holding the relevant public key (the only exception is SendDisputeOpen when sending to a moderator).
The most obvious benefit of this is that a vendor can interact with a buyer who is not publishing their public key to ipfs; all interactions with the buyer now use the public key as provided in the initial BuyerOrder.
The only functions still forced to do an ipfs lookup are:
SendChat, SendOfflineAck, Follow, Unfollow, all of which are not connected to a contract holding a public key.
And SendDisputeUpdate, which only ever goes to the moderator; moderator public keys are not stored in contracts.

Coverage decreased (-0.2%) to 27.292% when pulling d323abd61c00be0c3f30d471d254f2559223cfed on jackkleeman:contract-pubkey into e3878645419f24680d4f775c1bfc6abd26e1ddf2 on OpenBazaar:master.

moderator public keys are not stored in contracts

Is that something we should consider changing? cc @cpacia

@JustinDrake yes I think it would be ideal to have the contract store all information needed to complete the transaction without ipfs or any kind of internet connection needed except directly between the relevant parties.


FALCON-1867 hardcoded query names in JDBCStateStore

:+1:  LGTM. Will merge tomorrow unless someone has review comments.

The failure is unrelated to the patch. Merging it.


fix: Fix issue about loading Camera with the latest version

Pull Request Test Coverage Report for Build 867129439

-72 of 103 (30.1%)  changed or added relevant lines in 19 files are covered.
14 unchanged lines in 7 files lost coverage.
Overall coverage increased (+0.5%) to 22.222%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

packages/gphoto2-driver/src/components/PointerWrapper.ts
0
1
0.0%

packages/gphoto2-driver/src/components/Widget.ts
0
1
0.0%

packages/gphoto2-driver/src/components/CameraWidgets.ts
0
2
0.0%

packages/gphoto2-driver/src/index.ts
0
2
0.0%

packages/gphoto2-core/src/utils/GPUtils.ts
0
3
0.0%

packages/gphoto2-driver/src/utils/sleep.ts
0
3
0.0%

packages/gphoto2-driver/src/components/LiveView.ts
0
4
0.0%

packages/gphoto2-driver/src/components/AbilitiesList.ts
0
5
0.0%

packages/gphoto2-driver/src/components/CameraFile.ts
0
5
0.0%

packages/gphoto2-driver/src/components/PortInfoList.ts
0
5
0.0%

Files with Coverage Reduction
New Missed Lines
%

packages/gphoto2-driver/src/components/CameraWidgets.ts
1
0%

packages/gphoto2-driver/src/components/LiveView.ts
1
0%

packages/gphoto2-driver/src/components/CameraFile.ts
2
0%

packages/gphoto2-driver/src/components/CameraList.ts
2
0%

packages/gphoto2-driver/src/components/Camera.ts
2
0%

packages/gphoto2-driver/src/components/AbilitiesList.ts
3
0%

packages/gphoto2-driver/src/components/PortInfoList.ts
3
0%

Totals

Change from base Build 866209534:
0.5%

Covered Lines:
222

Relevant Lines:
952

💛  - Coveralls

:tada: This PR is included in version 3.0.3 :tada:
The release is available on:

v3.0.3
GitHub release

Your semantic-release bot :package::rocket:


Investigate if the HL7 serializer can handle other ORU R01 segments

Investigate if the HL7 serializer can handle other ORU R01 segments we do not need or use.  What happens when we get NK1 and PV1 segments?  Do we break the code?
Acceptance criteria:

Investigate what other segments are allowed in an ORU R01 message.
Investigate if any of the extra segments break the code and open issues if it does.
Document findings in this issue.

Tested the ingest with the attached file.   Test data has extra segments per the HL7 spec. No issues found with the HAPI parser.
HL7 ORU-R01 extra segments.txt


Moving groups of tiles in the 2D editor

Godot version:
3.0.6
OS/device including version:
Windows 8.1
Issue description:
Hi, in the 2D editor after creating a tilemap and putting a bunch of tiles in a level of the game, it's not possible to move a selection of tiles and reallocate that in another space but you need to erase and recreate it with what you want. Any chance to add this feature in 3.1 version? Thank you.
Steps to reproduce:
Create a tilemap, start to make a level for your game but you will need to erase the tiles in the 2D editor every time you change your mind about the level structure.
Minimal reproduction project:
Any level 1 Mario games.

3.1 is right around the corner, so not likely for it to happen in 3.1., rather in 3.2.

Thanks! I will wait for sure.

Actually that became an issue since 3.0 https://github.com/godotengine/godot/issues/16293

In 3.1beta3 selecting a group of tiles and then copy you get empty tiles.
The image comes from an atlas texture.
I had to redraw tiles manually.

Besides the "Cut, Copy, Erase Selection" there should be a "Move Selection", that would be useful. But to be honest, I think the TileMap editor in general is a bit unproductive.
I wish you could move and copy the tiles around as easily as regular nodes in the editor.

@AttackButton Sorry to post on an archived post, but is this issue just closed without turning it into a suggestion in godot-proposals?
The close reason is not clear, and as @Banduck said, the workaround requires 3 steps.

@hsandt Hmm! I think this is working since some versions ago. Isn't that working in 3.2?

@AttackButton it is working. I am using it in 3.2.1 stable.
I think that there should be another ticket with improvement suggestions, similar to what @Banduck said.


[BUG] Custom K3S image based on Ubuntu 20.04 fails with: error overwriting contents of /etc/hosts

What did you do
I'm building custom images for running cuda base workloads. When building a custom image based on Ubuntu 20.04, the process fails with the following error.
DEBU[0013] Executing command '[sh -c cat /tmp/-etc-hosts-qsSPeKgMSSKEwQIPZyiW > /etc/hosts]' in node 'k3d-gputest-server-0'

TRAC[0013] Exec process '[sh -c cat /tmp/-etc-hosts-qsSPeKgMSSKEwQIPZyiW > /etc/hosts]' still running in node 'k3d-gputest-server-0'.. sleeping for 1 second...

ERRO[0014] Failed Cluster Start: error during post-start cluster preparation: error overwriting contents of /etc/hosts: Exec process in node 'k3d-gputest-server-0' failed with exit code '126'

NOTE: This does not occur if the base image is changed to a variant of 18.04.
The failure appears to occur when running NewHostAliasesInjectEtcHostsAction:
https://github.com/rancher/k3d/blob/949e9c6c3295778d7c0b5545d8e5b84948973b36/pkg/actions/nodehooks.go#L125

How was the cluster created?
Using this Dockerfile create a custom image for K3S based on Ubuntu 20.04, build and run the image with k3d cluster create gputest --image=$IMAGE --gpus=all --trace --no-rollback.

This problem occurs even with the straight up base ubuntu:20.04 image from docker hub (i.e. no cuda stuff).
Note, this problem does not appear if you run on 18.04 versions of images.

What did you do afterwards?

What did you expect to happen
No failure
Which OS & Architecture

Linux

Which version of k3d

k3d version v5.3.0

Which version of docker
Docker Version
Client: Docker Engine - Community
Version:           20.10.12
API version:       1.41
Go version:        go1.16.12
Git commit:        e91ed57
Built:             Mon Dec 13 11:45:33 2021
OS/Arch:           linux/amd64
Context:           default
Experimental:      true
Server: Docker Engine - Community
Engine:
Version:          20.10.12
API version:      1.41 (minimum version 1.12)
Go version:       go1.16.12
Git commit:       459d0df
Built:            Mon Dec 13 11:43:42 2021
OS/Arch:          linux/amd64
Experimental:     false
containerd:
Version:          1.4.12
GitCommit:        7b11cfaabd73bb80907dd23182b9347b4245eb5d
nvidia:
Version:          1.0.2
GitCommit:        v1.0.2-0-g52b36a2
docker-init:
Version:          0.19.0
GitCommit:        de40ad0
Docker Info
Client:
Context:    default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.7.1-docker)
scan: Docker Scan (Docker Inc., v0.12.0)
Server:
Containers: 21
Running: 3
Paused: 0
Stopped: 18
Images: 280
Server Version: 20.10.12
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: false
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux nvidia runc
Default Runtime: nvidia
Init Binary: docker-init
containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc version: v1.0.2-0-g52b36a2
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.11.0-1028-azure
Operating System: Ubuntu 20.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 6
Total Memory: 110.1GiB
Name: remotegpu
ID: 26BB:RTMQ:7AX4:GYLP:LGNM:5T7B:D6FV:BPSJ:ZVIC:TDXD:CWJC:UBYY
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
<IP_ADDRESS>/8
Live Restore Enabled: false

This issue can be closed.
In the sample to get cuda running with k3d, it copies the base container over with COPY --from=k3s / / which overwrites everything - this is causing some issues with simple commands in the container like sh.

@jakkaj did you ever get this running with Ubuntu 20.04? I'm looking at creating a base image from Ubuntu

For future reference: https://github.com/k3d-io/k3d/discussions/989


fix(syslog-ng): Revert to 3.29.1 due to core dumps on 3.30.1

:tada: This PR is included in version 1.47.0-develop.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.47.0-develop.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


summarize chapters

Read the five chapters mentioned above. Each one in the group writes a short reflection (not more than one A4) on how the literature is applicable to your project and what insights they give on how to work in larger projects than this. End the reflection with one sentence per chapter describing the main take-away of that chapter. Post your reflections as comments to the issue created for this seminar. Post the reflections no later than midnight the day before the seminar.

@ErikWilhelmWohry, @Froding0, @rebeccahuynh0, @JakobSkoglund, and @SharbelKH
Thank you all. Nice reflections, well done!


accept a bee debug api url as input

to allow connections to remote bees

fixed - https://github.com/rampall/doctor-bee/commit/b612728e416e0069416232ddffeb159a1fc070d8


HyperLogLogPlusAggregator only works with one size of sketch

The HyperLogLogPlusAggregator only works with sketches where p = 5 and sp = 5.
This behaviour can be seen by changing the two instances of new HyperLogLogPlus(5, 5) in the setup method of HyperLogLogPlusAggregatorTest to new HyperLogLogPlus(6, 6). The test then fails.
The problem is caused by the following line in HyperLogLogPlusAggregator:
if (null == sketch) {
    sketch = new HyperLogLogPlus(5, 5);
}

As the sketch variable is always null after the aggregator is initialised, this means that the first sketch to be aggregated is merged into the HyperLogLogPlus with parameters 5 and 5. If the sketch to be merged in has different parameters then it fails with a java.lang.RuntimeException: An Exception occurred when trying to aggregate the HyperLogLogPlus objects.
To fix this, when the first sketch is merged in, the sketch variable should be set to a clone of the first sketch.

Merged into develop.