imprecise line indicator

Description
Line information in analysis results is imprecise.
For example, for mythril/solidity_examples/suicide.sol, Mythril reports line 5 is vulnerable,  but the problematic line is 7.
How to Reproduce
Assuming Mythril's Docker container is running and the file to be analyzed is in `/tmp/`` directory of the container, my command was:
sudo docker exec CONTAINER_NAME myth analyze /tmp/suicide.sol

Environment
I tested using the latest version of Docker image (v.0.21.20).

Mythril version: v.0.21.20
OS and Version: Ubuntu 18.04.2 (Host)

@SunBeomSo It seems to work fine for me, can you check whether your file was altered any way. Also can you mention whether you are using the docker-dev container or the standard one?

@norhh
I am really sorry for my mistake. I tested the code after removing top two lines (blanks).
However, I sometimes encounter imprecise line information when analyzing other contracts using Mythril.
Here is a concrete example contract:
https://etherscan.io/address/0x13873a43694f039d18107afeac917aff6bc4fa96#code
My command was:
sudo docker exec CONTAINER_NAME myth analyze /tmp/FILE_NAME.sol:ERC20_ICO --execution-timeout 600 --solv 0.4.24

then Mythril outputs a following alarm (similarly, I removed top four blanks in the contract for testing).  However, as you can see, address[] public investorlist is not an assert statement.
==== Exception State ====
SWC ID: 110
Severity: Low
Contract: ERC20_ICO
Function name: investorlist(uint256)
PC address: 3725
Estimated Gas Usage: 944 - 1039
A reachable exception has been detected.
It is possible to trigger an exception (opcode 0xfe). Exceptions can be caused by type errors, division by zero, out-of-bounds array access, or assert violations. Note that explicit `assert()` should only be used to check invariants. Use `require()` for regular input checking.
--------------------
In file: /tmp/2018-13536.sol:244

address[] public investorlist

--------------------
Initial State:

Account: [CREATOR], balance: 0x1, nonce:0, storage:{}
Account: [ATTACKER], balance: 0x0, nonce:0, storage:{}
Account: [SOMEGUY], balance: 0x0, nonce:0, storage:{}

Transaction Sequence:

Caller: [CREATOR], data: [CONTRACT_CREATION], value: 0x0
Caller: [CREATOR], function: investorlist(uint256), txdata: 0xc6f7b7eb, value: 0x0

Also can you mention whether you are using the docker-dev container or the standard one?

I am not sure what the docker-dev container is. I obtained image via docker pull mythril/myth command.

Actually it's possible to access a public array of a smart contract, in that case a call with an out of bounds index would cause an assertion failure, It's just a false positive.

@norhh  If my understanding is correct, I think Mythril should have pinpointed the following statement:
investorlist.push(receiver) -1 // line 274 in the link above

because this statement is the only one that accesses elements of the array investorlist.
address[] public investorlist is just a declaration statement.

It is true positive in the sense of triggering an exception. By false positive I mean it is not a security vulnerability and nothing is wrong anywhere.

Thanks, I close the issue.


Build breaks: error[E0282]: type annotations needed for Box<_>

log
rust-1.80.1
FreeBSD 14.1

Fixed by #202


Cannot upload large files to Pachyderm with the s3_out feature

What happened?:
Summary:
When a Pachyderm pipeline exposes the output repository as an S3 endpoint, small files can be uploaded successfully but a large file gives the following error:
boto3.exceptions.S3UploadFailedError: Failed to upload ./output/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581 to out/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): output commit 0626f96700b64db9bae9dcca4e77d363 not finished

Details:
I start a Kubeflow Pipelines pipeline from Pachyderm that trains a model and is supposed to upload the trained model artifacts back to Pachyderm. The kubeflow pipeline uploads several small files with no issues, but when it gets to the large file the boto3 client says the upload failed due to an internal error:
(pls ignore that s3://out - it's missing the actual S3 endpoint address generated by Pachyderm and out is the name of the bucket)
INFO:root:Training succeeded!
INFO:root:Uploading... ./output to s3://out/
INFO:root:-- ./output -> out: Uploading... file ./output/eval.tf_record to s3://out/eval.tf_record
INFO:root:-- ./output -> out: Uploaded file ./output/eval.tf_record to s3://out/eval.tf_record in 0.15s
INFO:root:-- ./output -> out: Uploading... file ./output/train.tf_record0 to s3://out/train.tf_record0
INFO:root:-- ./output -> out: Uploaded file ./output/train.tf_record0 to s3://out/train.tf_record0 in 0.24s
INFO:root:-- ./output -> out: Uploading... file ./output/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581 to s3://out/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581
ERROR:root:Failed to run training
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/transfer.py", line 279, in upload_file
    future.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 265, in result
    raise self._exception
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 364, in _main
    **extra_args)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 676, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): output commit 0626f96700b64db9bae9dcca4e77d363 not finished
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/workspace/bert/run_training_from_s3.py", line 147, in <module>
    output_bucket=args.output_bucket
  File "/workspace/bert/run_training_from_s3.py", line 105, in run_training
    upload_dir_to_bucket(s3.Bucket(output_bucket), './output')
  File "/workspace/bert/run_training_from_s3.py", line 47, in upload_dir_to_bucket
    Key=bucket_key
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 209, in bucket_upload_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 131, in upload_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/transfer.py", line 287, in upload_file
    filename, '/'.join([bucket, key]), e))
boto3.exceptions.S3UploadFailedError: Failed to upload ./output/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581 to out/events.out.tfevents.1617697593.train-bert-on-kubeflow-bjjkx-1877627581: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): output commit 0626f96700b64db9bae9dcca4e77d363 not finished

The implementation of the upload_dir_to_bucket used above (you can see that in the traceback):
def upload_dir_to_bucket(s3_bucket, local_dir, prefix=''):
    logging.info(f'Uploading... {local_dir} to s3://{s3_bucket.name}/{prefix}')
    upload_start = time.time()

    for root, dirs, files in os.walk(local_dir):
        for filename in files:
            file_path = os.path.join(root, filename)
            file_relpath = os.path.relpath(file_path, local_dir)
            bucket_key = os.path.join(prefix, file_relpath)
            logging.info(f'-- {local_dir} -> {s3_bucket.name}: '
                         f'Uploading... file {file_path} to s3://{s3_bucket.name}/{bucket_key}')
            file_upload_start = time.time()
            s3_bucket.upload_file(
                Filename=file_path,
                Key=bucket_key
            )
            file_upload_elapsed = round(time.time() - file_upload_start, 2)
            logging.info(f'-- {local_dir} -> {s3_bucket.name}: '
                         f'Uploaded file {file_path} to s3://{s3_bucket.name}/{bucket_key} in {file_upload_elapsed}s')

    upload_elapsed = round(time.time() - upload_start, 2)
    logging.info(f'Uploaded {local_dir} to bucket s3://{s3_bucket.name}/{prefix} in {upload_elapsed}s')

The S3 client is created as follows:
    s3 = boto3.resource(
        service_name='s3',
        endpoint_url=s3_endpoint,
        aws_access_key_id='',
        aws_secret_access_key='',
        config=botocore.config.Config(
            read_timeout=3600,
            connect_timeout=900
        )
    )

The commit 0626f96700b64db9bae9dcca4e77d363 of the train-bert pipeline was not finished by that time indeed - the commit is not finished until the pipeline is completed. (it's finished now, though):
oleh@oleh-virtualbox:~$ pachctl list commit train-bert
REPO       BRANCH COMMIT                           FINISHED    SIZE   PROGRESS DESCRIPTION
train-bert master 0626f96700b64db9bae9dcca4e77d363 7 hours ago 877KiB -

My guess: boto3 client uses CompleteMultipartUpload operation that the Pachyderm's S3 server interprets as the command to do something with the current commit, and that "something" cannot be done until the commit is closed. But the commit will only get closed after the whole pipeline finishes.
Anyway, I was not able to find anything like this online, especially that the boto3 client gives a Pachyderm error string (about the unfinished commit) in the exception. What makes things even more interesting is that smaller files are uploaded successfully!
oleh@oleh-virtualbox:~$ pachctl list file train-bert@0626f96700b64db9bae9dcca4e77d363
NAME              TYPE SIZE
/eval.tf_record   file 155.7KiB
/train.tf_record0 file 721.3KiB

Pipeline specification file:
{
  "pipeline": {
    "name": "train-bert"
  },
  "description": "Train BERT model on Kubeflow Pipelines",
  "transform": {
    "cmd": [
      "/bin/bash"
    ],
    "stdin": [
      "python /workspace/bert/train_on_kubeflow.py --base-image containerhub.internal.org:7777/bert_trainer:v18 --kfp-host ml-pipeline.kubeflow.svc.cluster.local:8888 --train-data-bucket train-data --pretrained-model-bucket pretrained-model --output-bucket out"
    ],
    "image": "containerhub.internal.org:7777/bert_trainer:v18",
    "debug": false
  },
  "resource_requests": {
    "memory": "8G",
    "cpu": 2
  },
  "standby": false,
  "enable_stats": false,
  "s3_out": true,
  "datum_tries": 1,
  "input": {
    "cross": [
      {
        "pfs": {
          "name": "train-data",
          "repo": "train-test-split",
          "branch": "trigger",
          "glob": "/",
          "lazy": false,
          "s3": true
        }
      },
      {
        "pfs": {
          "name": "pretrained-model",
          "repo": "pretrained_bert",
          "glob": "/",
          "lazy": false,
          "s3": true
        }
      }
    ]
  }
}

What you expected to happen?:
The large file is uploaded to Pachyderm's S3 endpoint successfully, just like the small files.
How to reproduce it (as minimally and precisely as possible)?:
Minimal reproducible example.
main.py
import os
import sys
import time
import argparse
import logging

import boto3
import botocore

def create_large_file(out_filename):
    chunk_size = 1024 * 1024
    chunk_num = 1024 * 2  # 2 GiB
    logging.info(f'Starting creating {out_filename}')
    with open('/dev/urandom', 'rb') as randfile, open(out_filename, 'wb') as outfile:
        for chunk in range(chunk_num):
            outfile.write(randfile.read(chunk_size))
        outfile.flush()
    filesize_gib = os.stat(out_filename).st_size / 1024 / 1024 / 1024
    logging.info(f'Done writing {out_filename} file: {filesize_gib} GiB')

def main():
    create_large_file('outfile')

    s3 = boto3.resource(
        service_name='s3',
        endpoint_url=os.environ['S3_ENDPOINT'],
        aws_access_key_id='',
        aws_secret_access_key='',
        config=botocore.config.Config(
            read_timeout=3600,
            connect_timeout=900
        )
    )

    logging.info('Starting uploading file')
    s3.Bucket('out').upload_file(Filename='outfile', Key='outfile')
    logging.info('Uploaded the file')

if __name__ == '__main__':
    logging.basicConfig(level=logging.INFO)

    try:
        main()
    except Exception:
        logging.exception('Failed')
        sys.exit(1)

Dockerfile
FROM python:3.6

WORKDIR /workspace

RUN pip3 install -Iv boto3~=1.17.0

COPY main.py .

test.pipeline
{
  "pipeline": {
    "name": "test-s3-out"
  },
  "description": "Test S3 Out feature with large files",
  "transform": {
    "cmd": [
      "/bin/bash"
    ],
    "stdin": [
      "python /workspace/main.py"
    ],
    "image": "containerhub.internal.org:7777/test:v3",
    "debug": false
  },
  "resource_requests": {
    "memory": "8G",
    "cpu": 2
  },
  "standby": false,
  "enable_stats": false,
  "s3_out": true,
  "datum_tries": 1,
  "input": {
    "cron": {
      "name": "test",
      "spec": "@every 1h"
    }
  }
}

Steps
version=v3
docker image build -t "containerhub.internal.org:7777/test:${version}" .
docker image push "containerhub.internal.org:7777/test:${version}"

pachctl create pipeline -f test.pipeline
pachctl run cron test-s3-out

Logs from the worker
{"pipelineName":"test-s3-out","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:27.780670585Z","message":"started transform spawner process"}
{"pipelineName":"test-s3-out","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:27.784933894Z","message":"transform spawner started"}
{"pipelineName":"test-s3-out","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.001790014Z","message":"created new job \"c77a0a969bdf4cc2aa3eead8c7cca6b0\" for output commit \"1aae03ab9a4840edb27b384d68c86115\""}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.017903484Z","message":"started waiting for job inputs"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.023951667Z","message":"finished waiting for job inputs"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.023996714Z","message":"started constructing datum iterator"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.065758872Z","message":"finished constructing datum iterator"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.065852977Z","message":"updating job info, state: JOB_RUNNING"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.078378836Z","message":"started processing job datums"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.078447321Z","message":"processJobRunning creating task channel"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.078501824Z","message":"started collecting datums for tasks"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.078501824Z","message":"started collecting datums for tasks"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:23:45.111683215Z","message":"finished collecting datums for tasks"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:23:45.132194070Z","message":"started datum task"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:23:45.135559714Z","message":"checking s3 gateway service for job \"c77a0a969bdf4cc2aa3eead8c7cca6b0\": \u003cnil\u003e"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:23:45.135806629Z","message":"transform worker datum task: job_id:\"c77a0a969bdf4cc2aa3eead8c7cca6b0\" datums_object:\"artifacts/job-c77a0a969bdf4cc2aa3eead8c7cca6b0/chunk-datum-list-1cd511300d0c403cac5997a1f37ecedc\" output_commit:\u003crepo:\u003cname:\"test-s3-out\" \u003e id:\"1aae03ab9a4840edb27b384d68c86115\" \u003e "}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:23:45.135842001Z","message":"started processing datums"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-06T18:23:45.164091188Z","message":"starting to download data"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-06T18:23:45.206944889Z","message":"finished downloading data after 42.686389ms"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-06T18:23:45.207109879Z","message":"beginning to run user code"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:23:45.388042775Z","message":"INFO:root:Starting creating outfile"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:23:56.902455834Z","message":"INFO:root:Done writing outfile file: 2.0 GiB"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:23:57.020011494Z","message":"INFO:root:Starting uploading file"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.972817600Z","message":"ERROR:root:Failed"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973021286Z","message":"Traceback (most recent call last):"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973069698Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py\", line 279, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973102636Z","message":"    future.result()"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973147087Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/futures.py\", line 106, in result"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973180807Z","message":"    return self._coordinator.result()"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973213828Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/futures.py\", line 265, in result"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973242396Z","message":"    raise self._exception"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973269401Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py\", line 126, in __call__"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973297227Z","message":"    return self._execute_main(kwargs)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973323718Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py\", line 150, in _execute_main"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973372659Z","message":"    return_value = self._main(**kwargs)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973399251Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py\", line 364, in _main"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973431613Z","message":"    **extra_args)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973460781Z","message":"  File \"/usr/local/lib/python3.6/site-packages/botocore/client.py\", line 357, in _api_call"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973515434Z","message":"    return self._make_api_call(operation_name, kwargs)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973549697Z","message":"  File \"/usr/local/lib/python3.6/site-packages/botocore/client.py\", line 676, in _make_api_call"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973599161Z","message":"    raise error_class(parsed_response, operation_name)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973634427Z","message":"botocore.exceptions.ClientError: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): output commit 1aae03ab9a4840edb27b384d68c86115 not finished"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973677272Z"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973709286Z","message":"During handling of the above exception, another exception occurred:"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973739207Z"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973764742Z","message":"Traceback (most recent call last):"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973793519Z","message":"  File \"/workspace/main.py\", line 46, in \u003cmodule\u003e"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973820497Z","message":"    main()"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973846739Z","message":"  File \"/workspace/main.py\", line 38, in main"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973876209Z","message":"    s3.Bucket('out').upload_file(Filename='outfile', Key='outfile')"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973910999Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py\", line 209, in bucket_upload_file"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973940815Z","message":"    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.973975053Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py\", line 131, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.974006596Z","message":"    extra_args=ExtraArgs, callback=Callback)"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.974034121Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py\", line 287, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.974061098Z","message":"    filename, '/'.join([bucket, key]), e))"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-06T18:25:02.974090041Z","message":"boto3.exceptions.S3UploadFailedError: Failed to upload outfile to out/outfile: An error occurred (InternalError) when calling the CompleteMultipartUpload operation (reached max retries: 4): output commit 1aae03ab9a4840edb27b384d68c86115 not finished"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-06T18:25:03.018657159Z","message":"errored running user code after 1m17.811484314s: exit status 1"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","datumId":"afa31c4872ca98dbbe4b28cd0e58a60343d8c7e6fb5098aaa61737835a7ee1a0","data":[{"path":"/2021-04-06T18:23:44Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-06T18:25:03.019081534Z","message":"failed to process datum with error: exit status 1\ngithub.com/pachyderm/pachyderm/src/client/pkg/errors.EnsureStack\n\t/app/src/client/pkg/errors/errors.go:50\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).RunUserCode\n\t/app/src/server/worker/driver/driver.go:782\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func<IP_ADDRESS>\n\t/app/src/server/worker/pipeline/transform/worker.go:463\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.(*Status).withDatum\n\t/app/src/server/worker/pipeline/transform/status.go:89\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2.1.1\n\t/app/src/server/worker/pipeline/transform/worker.go:461\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).WithActiveData\n\t/app/src/server/worker/driver/driver_unix.go:51\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2.1\n\t/app/src/server/worker/pipeline/transform/worker.go:455\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).WithData\n\t/app/src/server/worker/driver/driver.go:502\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2\n\t/app/src/server/worker/pipeline/transform/worker.go:452\ngithub.com/pachyderm/pachyderm/src/server/pkg/backoff.RetryUntilCancel\n\t/app/src/server/pkg/backoff/retry.go:113\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum\n\t/app/src/server/worker/pipeline/transform/worker.go:448\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.handleDatumTask.func<IP_ADDRESS>.1\n\t/app/src/server/worker/pipeline/transform/worker.go:311\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1374"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:25:03.019146137Z","message":"finished processing datums"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","ts":"2021-04-06T18:25:03.019205924Z","message":"finished datum task"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:25:03.031046588Z","message":"updating job info, state: JOB_RUNNING"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:25:03.047170843Z","message":"finished running datum tasks"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:25:03.047231074Z","message":"failing job with reason: datum failed"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:25:03.127975103Z","message":"Cleaning job artifacts"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:28:36.988615213Z","message":"finished processing job datums"}
{"pipelineName":"test-s3-out","jobId":"c77a0a969bdf4cc2aa3eead8c7cca6b0","workerId":"pipeline-test-s3-out-v1-xc8gf","master":true,"ts":"2021-04-06T18:28:36.988849174Z","message":"master done running processJobs"}

The error above is the same as I got when uploading the data from Kubeflow.
Anything else we need to know?:
Environment?:

Kubernetes version (use kubectl version):

Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.1", GitCommit:"c4d752765b3bbac2237bf87cf0b1c2e307844666", GitTreeState:"clean", BuildDate:"2020-12-18T12:09:25Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.14", GitCommit:"f238f5142728be4033c37aa0ad69bf806090beae", GitTreeState:"clean", BuildDate:"2020-11-11T13:03:54Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}

Pachyderm CLI and pachd server version (use pachctl version):

COMPONENT           VERSION
pachctl             1.12.2
pachd               1.12.4

Cloud provider (e.g. aws, azure, gke) or local deployment (e.g. minikube vs dockerized k8s): None, on-premises.
OS (e.g. from /etc/os-release):
Others:

root@fdce2903913a:/workspace/bert# python
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
>>> boto3.__version__
'1.17.36'

/kind bug
Added a minimal reproducible example.
Turns out this problem even doesn't need Kubeflow to reproduce.
In the minimal example, a 2 GiB file is created and uploaded to the S3 out repository from the Pachyderm worker itself.
The upload fails with exactly the same error as I had in a Kubeflow setup.

A workaround courtesy of @pappasilenus : disable multipart upload by:

using client.put_object() what never uses multipart or
increase the multipart threshold in the TransferConfig object to avoid using multipart.

The workaround is described in this SO issue.
Unfortunately, it didn't work in my case because without MultiPart upload boto3 seems to have an upper limit on the file size it's uploading. I got this error:
boto3.exceptions.S3UploadFailedError: Failed to upload outfile to out/outfile: An error occurred (EntityTooLarge) when calling the PutObject operation: Your proposed upload exceeds the maximum allowed object size.

Details
I've modified by Minimal Reproducible example from above as follows:
main.py
import os
import sys
import time
import argparse
import logging

import boto3
import botocore
from boto3.s3.transfer import TransferConfig

def create_large_file(out_filename):
    chunk_size = 1024 * 1024
    chunk_num = 1024 * 2  # 2 GiB
    logging.info(f'Starting creating {out_filename}')
    with open('/dev/urandom', 'rb') as randfile, open(out_filename, 'wb') as outfile:
        for chunk in range(chunk_num):
            outfile.write(randfile.read(chunk_size))
        outfile.flush()
    filesize_gib = os.stat(out_filename).st_size / 1024 / 1024 / 1024
    logging.info(f'Done writing {out_filename} file: {filesize_gib} GiB')

def main():
    create_large_file('outfile')

    transfer_config = TransferConfig(  # new
        multipart_threshold=9999999999999999
#        multipart_chunksize=10485760,
#        num_download_attempts=10,
#        use_threads=False
    )

    s3 = boto3.resource(
        service_name='s3',
        endpoint_url=os.environ['S3_ENDPOINT'],
        aws_access_key_id='',
        aws_secret_access_key='',
        config=botocore.config.Config(
            read_timeout=3600,
            connect_timeout=900
        )
    )

    logging.info('Starting uploading file')
    s3.Bucket('out').upload_file(Filename='outfile', Key='outfile', Config=transfer_config)  # updated
    logging.info('Uploaded the file')

if __name__ == '__main__':
    logging.basicConfig(level=logging.INFO)

    try:
        main()
    except Exception:
        logging.exception('Failed')
        sys.exit(1)

Note the use of the TransferConfig object and the multipart_threshold parameter.
Running the code exactly as described above produces the following logs:
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","ts":"2021-04-12T07:52:08.552108786Z","message":"started processing datums"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-12T07:52:08.583623059Z","message":"starting to download data"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-12T07:52:08.633399559Z","message":"finished downloading data after 49.674604ms"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-12T07:52:08.633529126Z","message":"beginning to run user code"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:08.846406688Z","message":"INFO:root:Starting creating outfile"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:20.553994852Z","message":"INFO:root:Done writing outfile file: 2.0 GiB"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:20.622061138Z","message":"INFO:root:Starting uploading file"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127712846Z","message":"ERROR:root:Failed"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127830136Z","message":"Traceback (most recent call last):"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127868364Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py\", line 279, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127902641Z","message":"    future.result()"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127935554Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/futures.py\", line 106, in result"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127969040Z","message":"    return self._coordinator.result()"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.127997420Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/futures.py\", line 265, in result"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128026089Z","message":"    raise self._exception"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128056038Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py\", line 126, in __call__"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128085722Z","message":"    return self._execute_main(kwargs)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128124694Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py\", line 150, in _execute_main"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128157215Z","message":"    return_value = self._main(**kwargs)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128204578Z","message":"  File \"/usr/local/lib/python3.6/site-packages/s3transfer/upload.py\", line 692, in _main"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128241905Z","message":"    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128275454Z","message":"  File \"/usr/local/lib/python3.6/site-packages/botocore/client.py\", line 357, in _api_call"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128306273Z","message":"    return self._make_api_call(operation_name, kwargs)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128334786Z","message":"  File \"/usr/local/lib/python3.6/site-packages/botocore/client.py\", line 676, in _make_api_call"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128369310Z","message":"    raise error_class(parsed_response, operation_name)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128398569Z","message":"botocore.exceptions.ClientError: An error occurred (EntityTooLarge) when calling the PutObject operation: Your proposed upload exceeds the maximum allowed object size."}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128435166Z"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128462627Z","message":"During handling of the above exception, another exception occurred:"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128495689Z"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128524579Z","message":"Traceback (most recent call last):"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128553416Z","message":"  File \"/workspace/main.py\", line 54, in \u003cmodule\u003e"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128584928Z","message":"    main()"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128612769Z","message":"  File \"/workspace/main.py\", line 46, in main"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128641752Z","message":"    s3.Bucket('out').upload_file(Filename='outfile', Key='outfile', Config=transfer_config)  # updated"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128672233Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py\", line 209, in bucket_upload_file"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128712793Z","message":"    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128753371Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py\", line 131, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128792676Z","message":"    extra_args=ExtraArgs, callback=Callback)"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128837293Z","message":"  File \"/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py\", line 287, in upload_file"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128864914Z","message":"    filename, '/'.join([bucket, key]), e))"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"user":true,"ts":"2021-04-12T07:52:29.128894805Z","message":"boto3.exceptions.S3UploadFailedError: Failed to upload outfile to out/outfile: An error occurred (EntityTooLarge) when calling the PutObject operation: Your proposed upload exceeds the maximum allowed object size."}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-12T07:52:29.162443931Z","message":"errored running user code after 20.52887078s: exit status 1"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","datumId":"674d2fbc5332e407ca77100a2d71f4381e7e437265307d7620e366d7283c176d","data":[{"path":"/2021-04-12T07:51:37Z","hash":"KlBJ7UW0cAVPljuSKQXXe6fL0GdOv4omOcVx5+scGXQ="}],"ts":"2021-04-12T07:52:29.162720971Z","message":"failed to process datum with error: exit status 1\ngithub.com/pachyderm/pachyderm/src/client/pkg/errors.EnsureStack\n\t/app/src/client/pkg/errors/errors.go:50\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).RunUserCode\n\t/app/src/server/worker/driver/driver.go:782\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func<IP_ADDRESS>\n\t/app/src/server/worker/pipeline/transform/worker.go:463\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.(*Status).withDatum\n\t/app/src/server/worker/pipeline/transform/status.go:89\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2.1.1\n\t/app/src/server/worker/pipeline/transform/worker.go:461\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).WithActiveData\n\t/app/src/server/worker/driver/driver_unix.go:51\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2.1\n\t/app/src/server/worker/pipeline/transform/worker.go:455\ngithub.com/pachyderm/pachyderm/src/server/worker/driver.(*driver).WithData\n\t/app/src/server/worker/driver/driver.go:502\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum.func2\n\t/app/src/server/worker/pipeline/transform/worker.go:452\ngithub.com/pachyderm/pachyderm/src/server/pkg/backoff.RetryUntilCancel\n\t/app/src/server/pkg/backoff/retry.go:113\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.processDatum\n\t/app/src/server/worker/pipeline/transform/worker.go:448\ngithub.com/pachyderm/pachyderm/src/server/worker/pipeline/transform.handleDatumTask.func<IP_ADDRESS>.1\n\t/app/src/server/worker/pipeline/transform/worker.go:311\ngolang.org/x/sync/errgroup.(*Group).Go.func1\n\t/go/pkg/mod/golang.org/x/sync@v0.0.0-20201020160332-67f06af15bc9/errgroup/errgroup.go:57\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1374"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","ts":"2021-04-12T07:52:29.162800652Z","message":"finished processing datums"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","ts":"2021-04-12T07:52:29.162858849Z","message":"finished datum task"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","master":true,"ts":"2021-04-12T07:52:29.170486709Z","message":"updating job info, state: JOB_RUNNING"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","master":true,"ts":"2021-04-12T07:52:29.184644532Z","message":"finished running datum tasks"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","master":true,"ts":"2021-04-12T07:52:29.184705214Z","message":"failing job with reason: datum failed"}
{"pipelineName":"test-s3-out","jobId":"f7e3cc6f84fc4a74bbd867e132da53cc","workerId":"pipeline-test-s3-out-v1-qfp22","master":true,"ts":"2021-04-12T07:52:29.280808494Z","message":"Cleaning job artifacts"}

Something that came up in on call is that this issues doesn't seem to affect the global s3 gateway. @brycemcanally it was mentioned on the call that you had pinpointed why this was, is that accurate and could you document here where that split happens. It seems like the easiest path to fixing this is just going to be unifying those code paths, my guess as to why we split them is that the state is global. @msteffen I think you were the one who initially implemented the s3g in workers, do you remember making that split in code path?

The problem occurs when the s3 gateway is operating on commits with provenance. The storage format split is detected by either the commit having provenance or the commit being finished with an input format hashtree. For s3_out pipelines, the commits have provenance, which means that the only check is for finished input format hashtrees, which isn't the case when the commit is open. The multipart upload implementation relies on copy file from the open commit to coalesce the parts into one file, which requires reading from the open commit. The long term solution is unifying the storage formats, which will be the case with 2.0.


[WINGET] Packages with version " unknown" are listed in updates

Before submitting the issue

[x] I have checked that the bug is not a duplicate or, if it is, it needs to be reopened
[x] I have checked that I am running the latest version of WingetUI
[x] I have checked that this issue is not an issue with the winget utility, but rather with its implementation on WingetUI
[x] I have checked the FAQ section for solutions
[x] I have read the contributing guidelines and I agree with the Code of Conduct

Describe the issue
Apps with "unknown" version are shown in the "Updates" tab.
To Reproduce
Steps to reproduce the behaviour:

Install package with unknown reported version, such as Algodoo.
Check for updates.

The Log
My Log functionality is also not working. "Export to file" freezes the app and "Copy" just doesn't work, including the selection copy.
Additional context

If you update them, they will get registered as updated and won't show again. One of the new changes in the support for managing unknown apps

How do I register them as updated without actually updating them?

And how about the logging issue?

How do I register them as updated without actually updating them?

right click - > skip this version

And how about the logging issue?

which issue?

Pressing any "copy" button related to the logs doesn't work.
"Export log as file" hangs the entire app.

Can you please open a new issue for this? Thanks!


Chembl update

This is the updater for ChEMBL interactions.

The Chembl Updater claims grouped well (which is to be expected). All of the drug claims grouped (yay!) and all but 2 gene claims grouped (due to multi-alias match). All but 4 of the interaction claims grouped.

@yanyangfeng good catch. An association exists (https://github.com/griffithlab/dgi-db/blob/5f55aab0560b85f7a327a661f9aaf6635b26dc50/app/models/data_model/drug.rb#L10), but no FK is present in the database. I'll make a migration to add that.

closes #219

@yanyangfeng there are some fields (but not all) that we could add unique indices to, I'll make those changes as well to tighten up the tables a bit.


Bugs in the version <IP_ADDRESS>

When I select the mode of downloading images and meta-data of images in the details view, they are all downloaded at crawling, but in the details view the downloader value of images and metadata goes only in the images value, leaving the metadata value empty.
When I set the force rescan mode for the blogs, the download count in the column abnormally increases so it becomes greater than number total of images in the blogs and number images in the metadata. And after repeated forced scanning the values of download only increased.
After the forced scan with mode downloading metadata the program creates duplicate entries in the metadata files, in fact whole copies of meta-data are downloaded and created inside the text of the files of metadata.
Forced re-scanning does not work for deleted or renamed files, they do not download again. But it looks like all the missing files are downloaded after forced rescanning.

P.S. Sorry for possible mistakes in the text, English is not my native language

Thanks for sharing.


go1.5.1 to go1.5.3

1.5.1 is deprecated for this build back, deploy to bluemix button fails.

+1 thank you!!


Cannot set a token.

If you pass a token to the client as the readme documentation indicates:
client = Vonage::Client.new(token: token)
An error is thrown:
Uncaught exception: No application_id provided. Either provide an application_id, or set an auth token. You can add new applications from the Vonage dashboard.
If you pass in the application_id and private_key and the token, then the token parameter is ignored and a new token is generated.  I would like to be able to set a token.  If you do not want the user to be able to set a token, then you should remove that parameter or throw an error.
Gem version:
vonage 7.7.2

This is also issue form me. I'm not able to setup authentication.
I get the token like this and that works:
 claims = {
      application_id: app_id,
      private_key: private_key,
      nbf:<PHONE_NUMBER>,
      ttl: 800
    }

token  = Vonage::JWT.generate(claims)
client = Vonage::Client.new(token: token)

When I try to send the message I get this error:
No application_id provided. Either provide an application_id, or set an auth token. You can add new applications from the Vonage dashboard. See https://developer.nexmo.com/concepts/guides/applications for details, or email<EMAIL_ADDRESS>if you have any questions. (Vonage::AuthenticationError)

Initially I didn't want to go with JWT route at all, but other route also doesn't work for me:
client = Vonage::Client.new(application_id: app_id, private_key: private_key)

message = Vonage::Messaging::Message.whatsapp(type: 'text', message: message)
response = client.messaging.send(to: to, from: from, **message)

Error I get is:
{"title":"Unauthorised","detail":"Invalid Token"}
/Users/dixpac/.rbenv/versions/3.0.2/lib/ruby/gems/3.0.0/gems/vonage-7.7.2/lib/vonage/namespace.rb:213:in `parse': Vonage::AuthenticationError (Vonage::AuthenticationError)

I don't understand why I get the error, when same app_id and private_key gets me a token with a JWT approach.

@brendt-ujet thanks for raising this. I feel that this is a feature that should be in the SDK. At this stage I'm not sure whether the behaviour you've described is due to a bug or because the feature hasn't been fully/correctly implemented. I'll investigate and make any fixes necessary.

@dixpac what you're describing sounds like it might be a separate issue. Instantiating a client with app ID and private key should work (and was tested when implementing Messages API in the Ruby SDK):
client = Vonage::Client.new(application_id: app_id, private_key: private_key)

Can I just check, the Vonage Application to which the application_id and private_key relate:

Has been enabled for Messages v1?
Has a WhatsApp Business Account (WABA) number linked to the Application? (to check, go to External Accounts in the Dashboard, and see in the 'Application Usage' column for the WABA number if the Application name for the Application identified by the app id and private key is listed there.

This should now be fixed as part of the latest release. Closing this issue, but please re-open if you experience any further problems with this.


bookdown and rticles are required for manuscript

They are not required for the repro standard template:
In terminal:
docker run -it --rm rocker/verse:4.0.1 R

Then in terminal in R:
options(
	repos = c(aaronpeikert = 'https://aaronpeikert.r-universe.dev',
	CRAN = 'https://cloud.r-project.org')
)
install.packages('repro')
remove.packages("bookdown")
remove.packages("rticles") # is not there anyway

dir <- tempdir()
repro::use_repro_template(dir)
setwd(dir)
repro::automate()


Add a 9P debug interface

This PR allows a 9P filesystem to be exposed (over Unix domain socket or Windows named pipe) which allows the active network connections to be listed. For example:
 ~/mirage/ocaml-9p/_build/src/main.native ls connections --address unix:/tmp/slirp.socket
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/Users/djs/Library/Containers/com.docker.docker/Data/s50
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/tmp/slirp.socket
drwxrwxr-x ? root root 0 Jan 1 1970 tcp:<IP_ADDRESS>:443
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/Users/djs/Library/Containers/com.docker.docker/Data/s51
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/Users/djs/Library/Containers/com.docker.docker/Data/s40
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/Users/djs/Library/Containers/com.docker.docker/Data/s50
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/Users/djs/Library/Containers/com.docker.docker/Data/s51
drwxrwxr-x ? root root 0 Jan 1 1970 udp:<IP_ADDRESS>:41712-<IP_ADDRESS>:123
drwxrwxr-x ? root root 0 Jan 1 1970 unix:/tmp/slirp.socket

This uses the handy VFS library included with http://github.com/docker/datakit
Future patches could allow the connections to be manipulated e.g. rm to close.
This PR removes the SIGUSR1 interface which was tricky to use because it was a bit asynchronous.

You will probably need a release of datakit to make that work, I can make one later today.

Unfortunately the ocamlgraph download server seems broken too:
$ wget http://ocamlgraph.lri.fr/download/ocamlgraph-1.8.7.tar.gz
--2016-09-29 11:43:40--  http://ocamlgraph.lri.fr/download/ocamlgraph-1.8.7.tar.gz
Resolving ocamlgraph.lri.fr... <IP_ADDRESS>
Connecting to ocamlgraph.lri.fr|<IP_ADDRESS>|:80... connected.
HTTP request sent, awaiting response...

Appveyor build recorded "Cancelled 4 hours ago in 52 min 23 sec" -- perhaps there's just too much code being built here?

I've switched the dependency to datakit-client following the https://github.com/mirage/mirage-dev/pull/149 example. Hopefully this builds in time.

As expected, Fs9p is missing:
File "src/main.ml", line 100, characters 22-31:
Error: Unbound module Fs9p
Command exited with code 2.

Woo, build finally successful!

@djs55 I am releasing datakit-server today and I'll rebase your PR to use it directly.


Add inverse and simplify type structure

This PR implements the suggestion in https://github.com/JuliaGaussianProcesses/GPLikelihoods.jl/issues/43#issuecomment-920974828.
More concretely, if one defines GPLikelihoods.inverse(::typeof(g)) = f (probably one wants to define the reverse definition as well :wink:) then one gets inv(::Link{typeof(g)}) isa Link{typeof(f)} for free. This allows to define LogLink, ExpLink just as aliases of Link.
(As a hidden feature, in addition to existing links the PR also adds the suggestion in the linked comment for log1pexp and logexpm1, i.e., Link(log1pexp) and Link(logexpm1) are invertible now as well.)
In the future, maybe definitions for inv(exp) etc. might be available in base (https://github.com/JuliaLang/julia/issues/42421) or as type piracy in some package which would make this approach even more composable and put less maintenance burden on us.

I have to say I find them quite need.
I'd rather see Likelihood{<:LogisticLink} then Likelihood{<:Link{typeof(logistic)}} (which also implies one would need to have logistic in the workspace.

ToDo: Switch to InverseFunctions. ;-)

There's already an issue 😉
We can't switch before LogExpFunctions and StatsFuns support it though (if we don't want to introduce type piracy).


[Teams] Port TeamsInfo class

https://github.com/microsoft/botbuilder-dotnet/blob/master/libraries/Microsoft.Bot.Builder/Teams/TeamsInfo.cs

Done.


NotFoundError: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.

Please include some context and the contents of the console in your browser's developer tools.
JavaScript Error
Error: Failed to execute 'removeChild' on 'Node': The node to be removed is not a child of this node.
    at yl (http://<IP_ADDRESS>:8000/:146:141704)
    at Xp (http://<IP_ADDRESS>:8000/:146:251719)
    at Kp (http://<IP_ADDRESS>:8000/:146:252080)
    at gh (http://<IP_ADDRESS>:8000/:146:267692)
    at HTMLUnknownElement.v (http://<IP_ADDRESS>:8000/:146:103631)
    at Object.Bn (http://<IP_ADDRESS>:8000/:146:103956)
    at qn (http://<IP_ADDRESS>:8000/:146:104917)
    at mh (http://<IP_ADDRESS>:8000/:146:265326)
    at t.unstable_runWithPriority (http://<IP_ADDRESS>:8000/:146:38304)
    at Ws (http://<IP_ADDRESS>:8000/:146:148134)

Reproduction Steps
Please tell us what you were doing when this error occurred, so we can more easily debug it and find a solution.

…

User Agent
Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36

Thanks! This seems to be a duplicate of another issue on this repo. I'll close this issue for now and we'll continue this discussion there.


JSONDecodeError

I'm getting some problem using the esp_csi_tool.py, the issue is when we use a computer with poor I/O or cpu performance, for example our laptop without charger, there will be some problem reading the strings from serial port.

In the above pictures, obviously the data invalid, causing the whole frame lost.
In that case we can cope with the issue by using a more powerful PC, but it isn't just about the performance. esp_csi_tool.py writes each CSI_DATA into csi_data.csv, as the file getting larger, the handling speed will slow down and cause the issue above. A new version of esp_csi_tool.py is needed.

This is because the read and write speed of the serial port is too slow, resulting in data loss. At present, this problem has been fixed, please update

Hello, @JLnukeNV
Closing this issue, feel free to reopen with more updates. Thanks.


Error after running docker compose. It cannot complete the installation

Describe the bug
I get an error message in step 5/5
=> ERROR [perplexica-frontend 5/5] RUN yarn build
To Reproduce
Steps to reproduce the behavior:
run the command: docker compose up -d
Expected behavior

=> ERROR [perplexica-frontend 5/5] RUN yarn build                                                                                20.6s

[perplexica-frontend 5/5] RUN yarn build:
0.507 yarn run v1.22.19
0.525 $ next build
1.011 Attention: Next.js now collects completely anonymous telemetry regarding usage.
1.012 This information is used to shape Next.js' roadmap and prioritize features.
1.012 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
1.012 https://nextjs.org/telemetry
1.012
1.096    ▲ Next.js 14.1.4
1.096
1.153    Creating an optimized production build ...
2.602 (node:41) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
2.602 (Use node --trace-deprecation ... to show where the warning was created)
10.99 (node:139) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.
10.99 (Use node --trace-warnings ... to show where the warning was created)
11.33 (node:139) [DEP0040] DeprecationWarning: The punycode module is deprecated. Please use a userland alternative instead.
17.12  ✓ Compiled successfully
17.12    Linting and checking validity of types ...
20.43 Failed to compile.
20.43
20.43 ./components/EmptyChat.tsx:15:22
20.43 Type error: Type '{ size: number; className: string; }' is not assignable to type 'IntrinsicAttributes & { className?: string | undefined; }'.
20.43   Property 'size' does not exist on type 'IntrinsicAttributes & { className?: string | undefined; }'.
20.43
20.43   13 |   return (
20.43   14 |
20.43 > 15 |
20.43      |                      ^
20.43   16 |
20.43   17 |
20.43   18 |
20.47 error Command failed with exit code 1.
20.47 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

failed to solve: process "/bin/sh -c yarn build" did not complete successfully: exit code: 1
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

passing a prop called size to this component, which is not defined in its props type
add to themes/switcher.tsx
const ThemeSwitcher = ({ className, size }: { className?: string; size?: number }) => {...}
Fixed

+1

Same bug. The fix is working fine, thanks!

passing a prop called size to this component, which is not defined in its props type
modify in theme/switcher.tsx to read
const ThemeSwitcher = ({ className, size }: { className?: string; size?: number }) => {
Fixed

This worked. Thanks a lot !

This should probably be added as a PR right?


Feature request: erase_if

C++20 adds the member function templates std::set::erase_if and std::map::erase_if.
There are no gtl::btree_set::erase_if and gtl::btree_map::erase_if.
Please consider adding them.

My misunderstanding. It is a free function, not a member function.


Make the library signature more difficult to remove

Use compiler-specific mechanisms to prevent the removal of the identifying hash value and version number.
I've just stumbled over this while perusing the AFL++ source code and remembered this awkward, abused, unreachable exception.

Coverage remained the same at 100.0% when pulling f9594071aea70ec344bf086bde7126b6335a54d8 on falbrechtskirchinger:lib-signature into a8a547d7a212a6a39943bbd5b4220be504a1a33e on nlohmann:develop.

In the end, I couldn't quite make it work without redefinition issues.


Added Search algorithms in C++ folder of classical_algorithms

Your checklist for this pull request
🚨Please review the guidelines for contributing to this repository.

[x] Make sure your files are in the right folder!
[x] Make sure you add the extension of the file (.cpp, .python, .java, etc).
[x] If your code is a solution for a code problem, please tell which one.
[x] Make sure your code is working, test for a lot of cases.

Description
I have added Linear Search and Binary Search algorithms in C++ folder of classical_algorithms .

Can you please review and merge this PR under Hacktoberfest 2022 .

Also can you please assign hacktoberfest label to it .


feat: remove deprecated fields from localities autocomplete responses

Issue

https://github.com/Woosmap/localities/issues/965
Describe your changes

removed deprecated parameters descriptions from responses
How to test

Checklist:

[ ] My code follows the style guidelines for this repo
[ ] I have performed a self-review of my code
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings/errors
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I don't require ops changes for this PR to go to prod
[ ] This change does not include a migration

Comments

/update-dist


Call terminated when press hold button in Mozilla FireFox

Call terminated when i press hold button in Mozilla firefox latest stable update (48).
And in console i see this error: TypeError: this.o_pc.removeStream is not a function
What can i do?

i see this error too, i don't know what's causing it or how to fix it.
but it's enoying since's i need to make this work.

I am working only with "Mute" methods.

Same issue here, just works fine using a webkit based browser, any suggestion?


logs not found for nodes other than master

Environmental Info:
K3s Version:
(both nodes)
$ k3s -v
k3s version v1.30.3+k3s1 (f6466040)
go version go1.22.5

Node(s) CPU architecture, OS, and Version:

Master:

$ uname -a
Linux mima 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Agent:

# uname -a
Linux nile 5.15.0-119-generic #129-Ubuntu SMP Fri Aug 2 19:25:20 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: 1 master, 1 agent
Describe the bug:
Pod logs are not found on agent node. Master node works fine.
Steps To Reproduce:

Install K3s via curl -sfL https://get.k3s.io | sh - on master node.
Install K3s via curl -sfL https://get.k3s.io | K3S_URL=https://<IP_ADDRESS>:6443 K3S_TOKEN=mynodetoken sh - on agent node.
Ensure nodes is added:

NAME   STATUS   ROLES                  AGE   VERSION
mima   Ready    control-plane,master   45m   v1.30.3+k3s1
nile   Ready    <none>                 12m   v1.30.3+k3s1

Perform a deployment.

I deployed MySQL like this:
metadata:
  name: mysql
  namespace: metrics-demo
spec:
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.3.0
        env:
        - name: MYSQL_ROOT_PASSWORD
          value: "root"
        ports:
        - containerPort: 3306
        volumeMounts:
        - name: mysql-volume
          mountPath: /var/lib/mysql
      volumes:
        - name: mysql-volume
          persistentVolumeClaim:
            claimName: mysql-pvc
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mysql-pvc
  namespace: metrics-demo
spec:
  resources:
    requests:
      storage: 1Gi
  accessModes:
    - ReadWriteOnce
---

Wait for the pod to show up.

$ k3s kubectl get -n metrics-demo pods
NAME                                 READY   STATUS    RESTARTS      AGE
mysql-654948867-gjgns                1/1     Running   0             41s

Check the log.

$ k3s kubectl logs -n metrics-demo mysql-654948867-gjgns
Error from server (NotFound): the server could not find the requested resource ( pods/log mysql-654948867-gjgns)

Expected behavior:
Logs are shown.
Actual behavior:
Error from server like above.

Can you show the output of kubectl get node -o yaml ?
Something is returning a 404 when you try to get logs. Do you have a proxy configured in your environment? If so, have you properly configured the NO_PROXY environment variable to ensure the apiserver doesn't try the go through the proxy when requesting logs from the agent?

Both of your nodes have address: <IP_ADDRESS>. You could also see this very quickly in the output of kubectl get node -o wide. This is not a valid configuration, each node needs unique hostnames and addresses. If these nodes have multiple interfaces, try using the --node-ip option to set the correct address on each of them.
You should probably remove the NO_PROXY env vars if you do not have a proxy configured.


Fastify compatablity

Simple issue res.set causes fastify server to crash.
I just commented it out and everything works fine. But I am trying to repliment the res.set in fastify way.

If you come up with a solution to make it work for both PR would be very welcome...else might be a good time to create a i18next-fastify-middleware

I was thinking of that for now I just removed the res.set and copied it to my app. I am not using server url translation. I think thats broken now.

closing due inactivity


az login:  Failed to establish a new connection: [Errno -2] Name does not resolve

This is autogenerated. Please review and update as needed.
Describe the bug
When logging in with azure cli inside of mcr.microsoft.com/azure-cli:2.32.0 container I get an error (See below), The same login command works if I login outside of the container on my host.
Command Name
az login
Errors:
<urllib3.connection.HTTPSConnection object at 0x7f3ed60e6490>: Failed to establish a new connection: [Errno -2] Name does not resolve

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Are you running the docker container on WSL? If so, this could be related to

https://github.com/microsoft/WSL/issues/5991
https://github.com/microsoft/WSL/issues/5256

No I am running the docker container in a hyper-v Ubuntu 18.04 virtual machine.

I see. Then could you test if your container can resolve login.microsoftonline.com?
apt-get update
apt install dnsutils
nslookup login.microsoftonline.com

The result should look like
# nslookup login.microsoftonline.com
Server:         <IP_ADDRESS>
Address:        <IP_ADDRESS>#53

Non-authoritative answer:
login.microsoftonline.com       canonical name = ak.privatelink.msidentity.com.
ak.privatelink.msidentity.com   canonical name = www.tm.ak.prd.aadg.trafficmanager.net.
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>
Name:   www.tm.ak.prd.aadg.trafficmanager.net
Address: <IP_ADDRESS>

Here is the result:
bash-5.1# nslookup login.microsoftonline.com
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>:53

Non-authoritative answer:
login.microsoftonline.com	canonical name = ak.privatelink.msidentity.com
ak.privatelink.msidentity.com	canonical name = www.tm.ak.prd.aadg.akadns.net
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>
Name:	www.tm.ak.prd.aadg.akadns.net
Address: <IP_ADDRESS>

Non-authoritative answer:
login.microsoftonline.com	canonical name = ak.privatelink.msidentity.com
ak.privatelink.msidentity.com	canonical name = www.tm.ak.prd.aadg.akadns.net

Any update on this issue?

DNS resolution looks good. Could you check if Python can access https://login.microsoftonline.com/ ?
Use the Azure CLI image:
docker run -it mcr.microsoft.com/azure-cli
python -c "import requests; print(requests.get('https://login.microsoftonline.com/').status_code)"

Use official Python image based on Alpine:
docker run -it python:alpine
pip install requests
python -c "import requests; print(requests.get('https://login.microsoftonline.com/').status_code)"

However, I don't think this problem is caused by Azure CLI image. I would recommend checking the docker or Hyper-V configuration instead.

Yeah I can confirm I am having issues with https://login.microsoftonline.com/ now. Getting a different error message from last time on az login. Do you have any info what configuration is possibly causing this?
The command failed with an unexpected error. Here is the traceback:
HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /organizations/v2.0/.well-known/openid-configuration (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f83f3789240>: Failed to establish a new connection: [Errno -3] Try again'))
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/local/lib/python3.10/socket.py", line 955, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 382, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 358, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f83f3789240>: Failed to establish a new connection: [Errno -3] Try again

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /organizations/v2.0/.well-known/openid-configuration (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f83f3789240>: Failed to establish a new connection: [Errno -3] Try again'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/knack/cli.py", line 231, in invoke
    cmd_result = self.invocation.execute(args)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 658, in execute
    raise ex
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 721, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 692, in _run_job
    result = cmd_copy(params)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 328, in __call__
    return self.handler(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/command_modules/profile/custom.py", line 139, in login
    subscriptions = profile.login(
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/_profile.py", line 153, in login
    user_identity = identity.login_with_device_code(scopes=scopes, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/auth/identity.py", line 177, in login_with_device_code
    flow = self._msal_app.initiate_device_flow(scopes, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/cli/core/auth/identity.py", line 101, in _msal_app
    self._msal_app_instance = PublicClientApplication(self.client_id, **self._msal_app_kwargs)
  File "/usr/local/lib/python3.10/site-packages/msal/application.py", line 1466, in __init__
    super(PublicClientApplication, self).__init__(
  File "/usr/local/lib/python3.10/site-packages/msal/application.py", line 438, in __init__
    self.authority = Authority(
  File "/usr/local/lib/python3.10/site-packages/msal/authority.py", line 83, in __init__
    openid_config = tenant_discovery(
  File "/usr/local/lib/python3.10/site-packages/msal/authority.py", line 142, in tenant_discovery
    resp = http_client.get(tenant_discovery_endpoint, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/msal/individual_cache.py", line 269, in wrapper
    value = function(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='login.microsoftonline.com', port=443): Max retries exceeded with url: /organizations/v2.0/.well-known/openid-configuration (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f83f3789240>: Failed to establish a new connection: [Errno -3] Try again'))

@Svessir, I am afraid this is really out of my knowledge. 😥 Do other domains work, like https://github.com/, https://outlook.live.com/?

I saw a similar issue ERROR: <urllib3.connection.HTTPSConnection object at 0x7f8a18270610>: Failed to establish a new connection: [Errno -2] Name or service not known with calling  az storage blob upload

@evelyn-ys for awareness

Same issue here running the mcr.microsoft.com/azure-cli:latest container on rancher desktop

I've come across this a couple times when I switch my connection to an enterprise network. Restarting the container seems to fix the issue for me.

The issue seems to be with URL:
https://management.azure.com/tenants?api-version=2019-11-01
Steps to reproduce:
$ docker run -it --rm mcr.microsoft.com/azure-cli:latest
bash-5.1# curl https://management.azure.com/tenants?api-version=2019-11-01
curl: (6) Could not resolve host: management.azure.com

I ran into the same issue, and found out a workaround by adding the "--add-host=management.azure.com:<IP_ADDRESS>" to the docker command line.

I'm facing this error when testing my code in my company's Kubernetes cluster. Locally it works. The upload_blob method is crashing and giving me this error:
azure.core.exceptions.ServiceRequestError: <urllib3.connection.HTTPSConnection object at 0x7f5959eb3890>: Failed to establish a new connection: [Errno -2] Name or service not known
code:
`from azure.storage.blob import BlobServiceClient
import os
from typing import List, Tuple, ByteString
class AzureBlob:
def init(self):
self.blob_container_name = "example"
os.environ["HTTP_PROXY"] = "proxy.example"
self.container_client = self.create_service_client()
def create_service_client(self):
    az_connection_str = os.environ.get('azure_st_connection_string')

    blob_service_client = BlobServiceClient.from_connection_string(conn_str=az_connection_str )
    container_client = blob_service_client.get_container_client(container=self.blob_container_name)
    print("Client connected ...", flush=True)

    return container_client

def save_file(self, path: str, content: ByteString):
    blob = self.container_client.get_blob_client(path)
    blob.upload_blob(content)`

Duplicate of https://github.com/Azure/azure-cli/issues/16524