Failed to stress test download with vegeta

What versions are you running?

$ go list -m github.com/chromedp/chromedp
github.com/chromedp/chromedp v0.6.10
$ /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --version
Google Chrome 89.0.4389.128
$ go version
go version go1.16.3 darwin/amd64

What did you do? Include clear steps.
I wrote a service used to download files by chromedp and using vegeta to stress the service. You can find all of the service in the repo, but here’s the gist of it:
type Client struct {
  browserCtx context.Context
}

func NewClient() *Client {
  var (
    ctx    context.Context
    cancel context.CancelFunc
  )

  // create a new browser
  if os.Getenv("HEADLESS_ENV") == "container" {
    opts := []chromedp.ExecAllocatorOption{
      chromedp.ExecPath("/headless-shell/headless-shell"),
    }

    ctx, _ = chromedp.NewExecAllocator(context.Background(), opts...)
    ctx, cancel = chromedp.NewContext(ctx, chromedp.WithDebugf(log.Printf)) //chromedp.WithLogf(log.Printf))
  } else {
    ctx, cancel = chromedp.NewContext(context.Background(), chromedp.WithDebugf(log.Printf)) //chromedp.WithLogf(log.Printf))
  }

  if err := chromedp.Run(ctx); err != nil {
    cancel()
    log.Fatal("Failed to start browser:", err)
  }

  return &Client{
    browserCtx: ctx,
  }
}

func (c *Client) Download(ID, dlURL, dlDir string) error {
  var (
    errChan = make(chan error, 1)
    counter Counter
  )

  // Do something

  // Create a timeout as a safety net to prevent any infinite wait loops
  duration := 5 * time.Minute

  // create a new tab
  ctxTimeout, cancel := context.WithTimeout(c.browserCtx, duration)
  defer cancel()
  ctxTab, cancel := chromedp.NewContext(ctxTimeout)
  defer func() {
    cancel()
    if err := chromedp.Cancel(ctxTab); err != nil {
      err = errors.Wrapf(err, "ID:%s failed to close tab", ID)
      log.Println(err)
    }
  }()

  chromedp.ListenTarget(ctxTab, func(ev interface{}) {
    switch ev := ev.(type) {
    case *page.EventDownloadWillBegin:
      data, _ := ev.MarshalJSON()
      details := string(data)
      log.Printf("ID:%s download start:%s", ID, details)

      // Increase download counter when download start
      counter.Increase()

    case *page.EventDownloadProgress:
      data, _ := ev.MarshalJSON()
      details := string(data)

      switch ev.State {
      case page.DownloadProgressStateCanceled, page.DownloadProgressStateCompleted:
        if ev.State == page.DownloadProgressStateCanceled {
          log.Printf("ID:%s download canceled:%s", ID, details)
        } else {
          log.Printf("ID:%s download completed:%s", ID, details)
        }

        chromedp.Sleep(30 * time.Second)

        // Decrease download counter after download ended
        counter.Decrease()

        // Notify main routine download progress is done
        if counter.Value() == 0 {
          errChan <- nil
        }
      }
    }
  })

  err = chromedp.Run(
    ctxTab,
    page.SetDownloadBehavior(page.SetDownloadBehaviorBehaviorAllow).WithDownloadPath(dlDir),
    chromedp.Navigate(dlURL),
  )
  if err != nil && !strings.Contains(err.Error(), "net::ERR_ABORTED") {
    err = errors.Wrapf(err, "ID:%s failed to download file at URL %s", ID, dlURL)
    log.Println(err)
    return err
  }

  // Wait for the download to finish
  select {
  case <-ctxTimeout.Done():
    err = fmt.Errorf("Download not finish in %s", duration.String())
  case err = <-errChan:
  }

  if err != nil {
    err = errors.Wrapf(err, "ID:%s downloaded failed at URL:%s", ID, uri.RawQuery)
    log.Println(err)
  } else {
    // Check there are any files in download dir
    files, err := utils.ListFilesInDir(dlDir)
    if err == nil && len(files) > 0 {
      log.Printf("ID:%s, download dir:%s has %d files: %v", ID, dlDir, len(files), strings.Join(files, ","))
    } else {
      err = errors.Errorf("ID:%s not found any file in download dir: %s", ID, dlDir)
      log.Println(err)
      return err
    }
  }

  return err
}

After run vegeta to stress test service, i found some symptoms as follows. The full log file of items 1,2,3 is here

Download completed but no file in download dir.

2021/04/22 23:11:08 ID:571213 download completed:{"guid":"5274e4b9-4a08-42cb-a2c9-dbe6c7ce9e69","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/22 23:11:08 ID:571213 not found any file in download dir: /Users/n7n/chromedp_sample/download/571213

Download canceled but there is/are file(s) in download dir.

2021/04/22 23:10:41 ID:192614 download start:{"frameId":"DFBC0124ED6EC625FA16B02BEDAAEE53","guid":"545ef3da-9bae-4466-bdec-1dd6eed51b95","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/22 23:10:41 ID:192614 download canceled:{"guid":"545ef3da-9bae-4466-bdec-1dd6eed51b95","totalBytes":1.39460608e+08,"receivedBytes":0,"state":"canceled"}
2021/04/22 23:10:41 ID:192614, download dir:/Users/n7n/chromedp_sample/download/192614 has 5 files: /Users/n7n/chromedp_sample/download/192614/alpine-standard-3.13.5-x86_64.iso (1).crdownload,/Users/n7n/chromedp_sample/download/192614/alpine-standard-3.13.5-x86_64.iso (2).crdownload,/Users/n7n/chromedp_sample/download/192614/alpine-standard-3.13.5-x86_64.iso (3).crdownload,/Users/n7n/chromedp_sample/download/192614/alpine-standard-3.13.5-x86_64.iso (4).crdownload,/Users/n7n/chromedp_sample/download/192614/alpine-standard-3.13.5-x86_64.iso.crdownload

After download completed/canceled, these are some temporary files crdownload as above, then after some seconds, these files is disappear. I think temporary files are belong other download context/dirs, but some how, they are placed in download dir 192614 and after others's context close, these temporary files are deleted. This is the final output after some seconds:

$ tree download
download
├── 125254
├── 192614
│   └── alpine-standard-3.13.5-x86_64.iso
├── 251500
├── 416800
├── 458539
├── 571213
├── 646313
├── 827159
├── 872728
└── 874740
    └── alpine-standard-3.13.5-x86_64.iso

10 directories, 2 files

2021/04/22 23:12:10 ID:251500 download completed:{"guid":"52d80405-f290-4c98-b6db-c5e13339c9f3","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/22 23:12:10 ID:251500 not found any file in download dir: /Users/n7n/chromedp_sample/download/251500
2021/04/22 23:13:01 ID:416800 download completed:{"guid":"96fa6c32-5c39-411b-b7fd-793c7baf22fa","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/22 23:13:01 ID:416800 not found any file in download dir: /Users/n7n/chromedp_sample/download/416800
2021/04/22 23:14:26 ID:827159 downloaded failed at URL:: Download not finish in 5m0s
2021/04/22 23:14:27 ID:872728 downloaded failed at URL:: Download not finish in 5m0s

When I run this service in container, base on docker image chromedp/headless-shell, chromedp can download file with every single request. However, in case multiple simultaneous request, chromedp couldn't download any files in all requests and the output log is here

$ docker logs -f chromedp_sample
2021/04/22 17:31:56 ID:955411, browser has 1 tabs
2021/04/22 17:31:56 ID:771831, browser has 2 tabs
2021/04/22 17:31:56 ID:727096, browser has 3 tabs
2021/04/22 17:31:56 ID:672409, browser has 4 tabs
2021/04/22 17:31:57 ID:554455, browser has 5 tabs
2021/04/22 17:31:57 ID:801765, browser has 6 tabs
2021/04/22 17:31:57 ID:507485, browser has 7 tabs
2021/04/22 17:31:57 ID:296249, browser has 8 tabs
2021/04/22 17:31:57 ID:385934, browser has 9 tabs
2021/04/22 17:31:58 ID:645175, browser has 10 tabs
2021/04/22 17:31:58 ID:494854, browser has 11 tabs
2021/04/22 17:31:58 ID:612162, browser has 12 tabs
2021/04/22 17:31:58 ID:900839, browser has 13 tabs
2021/04/22 17:31:58 ID:984236, browser has 14 tabs
2021/04/22 17:31:59 ID:540736, browser has 15 tabs
2021/04/22 17:32:17 ID:955411 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:955411 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:507485 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:672409 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:507485 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:672409 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:727096 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:771831 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:771831 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:554455 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:554455 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:727096 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:296249 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:296249 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:900839 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:900839 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:540736 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:540736 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:385934 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:385934 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:612162 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:612162 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:494854 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:801765 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:645175 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:801765 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:645175 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:494854 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal
2021/04/22 17:32:17 ID:984236 received event EventTargetCrashed:{"targetId":"24E1555C7D2EBC6334A586B2F50D6018","status":"abnormal","errorCode":256}
2021/04/22 17:32:17 ID:984236 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: abnormal

What did you expect to see?
Obviously, I'd like to see chromedp can work well and has download file in correct download dir.
CC: @kenshaw, @mvdan, @ZekeLu, could you please give me some advice?
What did you see instead?

You encountered two issues here.

the number of downloaded files in the download directory is incorrect.
chromedp/headless-shell crashed under stress.

I want to answer the first question first. BTW, please don't ping mvdan. He is working on the golang project now, and should be very busy.
TL;DR, the scope of Page.setDownloadBehavior is browser-wide (instead of tab-wide). When multiple tabs set the download directory at the   same time, there is a race. That's not documented, but based on my test, this should be the reason.
Here is one of the scenarios:

tab 1 set the download directory to d1.
tab 2 set the download directory to d2.
tab 1 downloads the file to d2 (instead of d1)  with file name original-file-name.crdownload.
tab 2 downloads the file to d2 too with file name original-file-name (1).crdownload.
when tab 1 finish the downloading, it will rename original-file-name.crdownload to original-file-name.
when tab 2 finish the downloading, it will rename original-file-name (1).crdownload to original-file-name too (and just overwrite the existing file silently).

If you download different files from the server to avoid the overwriting, you will find some directory has multiple files. BTW, you should run a server (for example, http-server) to serve files locally instead of downloading from a public server. Be polite and don't spam public service.
FWIW, I found the root cause with strace. Here is the stripped-down logs from strace:
# run your service first, use `pgrep` to find out the process id of the browser, and pass it to `strace`
$ sudo strace -f -p 301934 2>&1 | tee trace.log
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 301949] access("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", F_OK) = -1 ENOENT (No such file or directory)
[pid 301949] creat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", 0666) = 82
[pid 301949] close(82)                  = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/.local/share/.com.google.Chrome.KNSpl1", {st_mode=S_IFREG|0600, st_size=41361408, ...}) = 0
[pid 301949] rename("~/.local/share/.com.google.Chrome.KNSpl1", "~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload") = 0
[pid 301949] chmod("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", 0100664) = 0
[pid 301949] openat(AT_FDCWD, "~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", O_RDWR) = 82
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 301949] access("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", F_OK) = -1 ENOENT (No such file or directory)
[pid 301949] creat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", 0666) = 91
[pid 301949] close(91)                  = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/.local/share/.com.google.Chrome.GqYar2", {st_mode=S_IFREG|0600, st_size=43757568, ...}) = 0
[pid 301949] rename("~/.local/share/.com.google.Chrome.GqYar2", "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload") = 0
[pid 301949] chmod("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", 0100664) = 0
[pid 301949] openat(AT_FDCWD, "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", O_RDWR) = 91
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 301949] access("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", F_OK) = -1 ENOENT (No such file or directory)
[pid 301949] creat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", 0666) = 78
[pid 301949] close(78)                  = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/.local/share/.com.google.Chrome.ajax52", {st_mode=S_IFREG|0600, st_size=44843008, ...}) = 0
[pid 301949] rename("~/.local/share/.com.google.Chrome.ajax52", "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload") = 0
[pid 301949] chmod("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", 0100664) = 0
[pid 301949] openat(AT_FDCWD, "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", O_RDWR) = 78
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 301949] access("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso", F_OK) = -1 ENOENT (No such file or directory)
[pid 301949] creat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso", 0666) = 80
[pid 301949] close(80)                  = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=139460608, ...}) = 0
[pid 301949] rename("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso.crdownload", "~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso") = 0
[pid 301949] chmod("~/src/chromedp/chromedp-sample/download/863591/alpine-standard-3.13.5-x86_64.iso", 0100664) = 0
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
[pid 301949] access("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", F_OK) = -1 ENOENT (No such file or directory)
[pid 301949] creat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", 0666) = 76
[pid 301949] close(76)                  = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", {st_mode=S_IFREG|0664, st_size=139460608, ...}) = 0
[pid 301949] rename("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso.crdownload", "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso") = 0
[pid 301949] chmod("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", 0100664) = 0
...
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187",  <unfinished ...>
[pid 301949] access("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso", F_OK) = 0
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso",  <unfinished ...>
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso",  <unfinished ...>
[pid 301949] stat("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload",  <unfinished ...>
[pid 301949] rename("~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso (1).crdownload", "~/src/chromedp/chromedp-sample/download/478187/alpine-standard-3.13.5-x86_64.iso" <unfinished ...>

I can not reproduce the second issue by simply downloading a file. According to your logs, it renders your own web page. And maybe your web page consumes too many resources. Try to give the container a larger shm-size to see whether it will become better. Refer to https://github.com/chromedp/docker-headless-shell#running for more information.

About the issue 1: the number of downloaded files in the download directory is incorrect.
With the previous source code (create every tab to handle each requests), I got confused when the symptoms occurs. Especially, received an event EventDownloadProgress with state page.DownloadProgressStateCanceled but still has a download file in dir.
Based on your comment, I'm understanding there is some bugs with downloading file in multiple tabs. I updated source code to create every browser instance for each requests and chromedp can download as expectation after running test.

Source code:

func (c *Client) DownloadWithBrowserCtx(ID, dlURL, dlDir string) error {
	// Do something

	// Create a timeout as a safety net to prevent any infinite wait loops
	duration := 5 * time.Minute

	// create a new browser
	ctxTimeout, cancel := context.WithTimeout(context.Background(), duration)
	defer cancel()

	var ctxBrowser context.Context
	if os.Getenv("HEADLESS_ENV") == "container" {
		opts := []chromedp.ExecAllocatorOption{
			chromedp.ExecPath("/headless-shell/headless-shell"),
		}

		ctxBrowser, _ = chromedp.NewExecAllocator(ctxTimeout, opts...)
		ctxBrowser, _ = chromedp.NewContext(ctxBrowser, chromedp.WithLogf(log.Printf)) // chromedp.WithDebugf(log.Printf))
	} else {
		ctxBrowser, _ = chromedp.NewContext(ctxTimeout, chromedp.WithLogf(log.Printf)) // chromedp.WithDebugf(log.Printf))
		// ctx, cancel = chromedp.NewContext(ctx, chromedp.WithDebugf(chromedpLog.Debugf)) // chromedp.WithLogf(h.log.Infof))
	}

	defer func() {
		if err := chromedp.Cancel(ctxBrowser); err != nil {
			err = errors.Wrapf(err, "ID:%s failed to close browser", ID)
			log.Println(err)
		}
	}()

	log.Printf("ID:%s, browser has %d tabs", ID, c.getNumOfTabs(ctxBrowser))
	err = chromedp.Run(
		ctxBrowser,
		page.SetDownloadBehavior(page.SetDownloadBehaviorBehaviorAllow).WithDownloadPath(dlDir),
		chromedp.Navigate(dlURL),
	)
	if err != nil && !strings.Contains(err.Error(), "net::ERR_ABORTED") {
		err = errors.Wrapf(err, "ID:%s failed to download file at URL %s", ID, dlURL)
		log.Println(err)
		return err
	}

	// ...
}

Output of download dir during service handling:

$ watch ls -Rlh ./download
Every 2.0s: ls -Rl ./download

./download:
total 36
drwxr-xr-x 2 root root 4096 Apr 23 11:50 141203
drwxr-xr-x 2 root root 4096 Apr 23 11:50 229780
drwxr-xr-x 2 root root 4096 Apr 23 11:50 243183
drwxr-xr-x 2 root root 4096 Apr 23 11:50 439849
drwxr-xr-x 2 root root 4096 Apr 23 11:50 597386
drwxr-xr-x 2 root root 4096 Apr 23 11:50 628816
drwxr-xr-x 2 root root 4096 Apr 23 11:50 655680
drwxr-xr-x 2 root root 4096 Apr 23 11:50 748923
drwxr-xr-x 2 root root 4096 Apr 23 11:50 938717

./download/141203:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/229780:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/243183:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/439849:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/597386:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/628816:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/655680:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/748923:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

./download/938717:
total 136196
-rw-r--r-- 1 root root 139460608 Apr 23 11:50 alpine-standard-3.13.5-x86_64.iso

Docker logs:

$ docker logs -f chromedp_sample
2021/04/23 11:49:36 ID:597386, browser has 1 tabs
2021/04/23 11:49:36 ID:597386 download start:{"frameId":"CFFDBD321730583AE078974896B75CD7","guid":"4d6d37eb-51f6-42c3-895d-f9d55a9fe09d","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:36 ID:243183, browser has 1 tabs
2021/04/23 11:49:37 ID:243183 download start:{"frameId":"90E2454CC21FE04711605CD88BA2E145","guid":"e3485400-9fab-42f1-8ce4-910d2acb8c51","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:37 ID:141203, browser has 1 tabs
2021/04/23 11:49:37 ID:748923, browser has 1 tabs
2021/04/23 11:49:37 ID:141203 download start:{"frameId":"852217062C69A7C7D0395F9BEDA9073C","guid":"c09b052f-c8df-42c9-8fe6-d2206d54eeff","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:37 ID:655680, browser has 1 tabs
2021/04/23 11:49:38 ID:748923 download start:{"frameId":"AAEC055B44C000CFF2120C933F2E4406","guid":"cbb146c2-952d-490e-865c-60e493a6ab30","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:38 ID:229780, browser has 1 tabs
2021/04/23 11:49:38 ID:655680 download start:{"frameId":"91BF9669DA5CB219DF7F4C2D6FBBD311","guid":"4d064c41-56bd-44ee-b8e5-923da5ffe780","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:38 ID:628816, browser has 1 tabs
2021/04/23 11:49:38 ID:229780 download start:{"frameId":"5291727A0599D4050C4C8D1A9CCB0D51","guid":"d0a37ab6-fc2b-4bf5-a294-ed442b77656e","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:38 ID:938717, browser has 1 tabs
2021/04/23 11:49:39 ID:628816 download start:{"frameId":"3921FAA0E7C212BA992DD53142533E16","guid":"2d9c3bf0-27f8-4d21-8ad2-5a962c907ceb","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:39 ID:439849, browser has 1 tabs
2021/04/23 11:49:39 ID:938717 download start:{"frameId":"9E34E88411C8A80110CD48DFCF73A210","guid":"481e41bf-4ed6-42e2-a024-4ab25fce55f6","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:49:40 ID:439849 download start:{"frameId":"7FEC21D2D6DE43961BF2BFFB2444B716","guid":"1fcdc3ba-6629-4825-b4b4-2e50fc87894a","url":"https://dl-cdn.alpinelinux.org/alpine/v3.13/releases/x86_64/alpine-standard-3.13.5-x86_64.iso","suggestedFilename":"alpine-standard-3.13.5-x86_64.iso"}
2021/04/23 11:50:09 ID:748923 download completed:{"guid":"cbb146c2-952d-490e-865c-60e493a6ab30","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:09 ID:748923, download dir:/app/download/748923 has 1 files: /app/download/748923/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:14 ID:938717 download completed:{"guid":"481e41bf-4ed6-42e2-a024-4ab25fce55f6","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:14 ID:938717, download dir:/app/download/938717 has 1 files: /app/download/938717/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:27 ID:655680 download completed:{"guid":"4d064c41-56bd-44ee-b8e5-923da5ffe780","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:27 ID:655680, download dir:/app/download/655680 has 1 files: /app/download/655680/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:29 ID:597386 download completed:{"guid":"4d6d37eb-51f6-42c3-895d-f9d55a9fe09d","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:29 ID:597386, download dir:/app/download/597386 has 1 files: /app/download/597386/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:33 ID:229780 download completed:{"guid":"d0a37ab6-fc2b-4bf5-a294-ed442b77656e","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:33 ID:229780, download dir:/app/download/229780 has 1 files: /app/download/229780/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:40 ID:439849 download completed:{"guid":"1fcdc3ba-6629-4825-b4b4-2e50fc87894a","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:40 ID:439849, download dir:/app/download/439849 has 1 files: /app/download/439849/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:41 ID:141203 download completed:{"guid":"c09b052f-c8df-42c9-8fe6-d2206d54eeff","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:41 ID:141203, download dir:/app/download/141203 has 1 files: /app/download/141203/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:42 ID:628816 download completed:{"guid":"2d9c3bf0-27f8-4d21-8ad2-5a962c907ceb","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:42 ID:628816, download dir:/app/download/628816 has 1 files: /app/download/628816/alpine-standard-3.13.5-x86_64.iso
2021/04/23 11:50:43 ID:243183 download completed:{"guid":"e3485400-9fab-42f1-8ce4-910d2acb8c51","totalBytes":1.39460608e+08,"receivedBytes":1.39460608e+08,"state":"completed"}
2021/04/23 11:50:43 ID:243183, download dir:/app/download/243183 has 1 files: /app/download/243183/alpine-standard-3.13.5-x86_64.iso

However, is it the good if we do as the above?

About the issue 2: chromedp/headless-shell crashed under stress.
I had use new code (new browser for each requests) and  increased size of shm to 2gb, 4gb and 8gb, but it still happens with new code and status (9 & killed)
$ docker exec -it chromedp_sample df -kh /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
shm             2.0G     0  2.0G   0% /dev/shm

$ docker logs -f chromedp_sample
2021/04/23 11:54:31 ID:862697, browser has 1 tabs
2021/04/23 11:54:32 ID:546094, browser has 1 tabs
2021/04/23 11:54:32 ID:158506, browser has 1 tabs
2021/04/23 11:54:32 ID:453343, browser has 1 tabs
2021/04/23 11:54:33 ID:973729, browser has 1 tabs
2021/04/23 11:54:33 ID:023296, browser has 1 tabs
2021/04/23 11:54:33 ID:161675, browser has 1 tabs
2021/04/23 11:54:34 ID:673276, browser has 1 tabs
2021/04/23 11:54:34 ID:577591, browser has 1 tabs
2021/04/23 11:55:32 ID:577591 received event EventTargetCrashed:{"targetId":"F7B56D00B07B8D31EB3647EC7675DBFD","status":"killed","errorCode":9}
2021/04/23 11:55:32 ID:577591 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 11:55:44 ID:453343 received event EventTargetCrashed:{"targetId":"7A3EBAB3BBE36A2FF4C72FD9155BAD5F","status":"killed","errorCode":9}
2021/04/23 11:55:44 ID:453343 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 11:55:51 ID:546094 received event EventTargetCrashed:{"targetId":"2CA025222634376AA7B387A30B857093","status":"killed","errorCode":9}
2021/04/23 11:55:51 ID:546094 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 11:56:07 ID:862697 received event EventTargetCrashed:{"targetId":"DE543609660BE44C003ECAA478229929","status":"killed","errorCode":9}
2021/04/23 11:56:07 ID:862697 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 11:56:17 ID:158506 download start:{"frameId":"6C9D93A66D103A2018ACE6AB9B024E58","guid":"cff87274-d51c-4f17-ae1f-f9e07f241e12","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/261770b5-391c-4c69-bdba-53f17c4f8c65","suggestedFilename":"0_file_0.zip"}
2021/04/23 11:56:18 ID:158506 download completed:{"guid":"cff87274-d51c-4f17-ae1f-f9e07f241e12","totalBytes":1.910861e+06,"receivedBytes":1.910861e+06,"state":"completed"}
2021/04/23 11:56:18 ID:158506, download dir:/app/download/158506 has 1 files: /app/download/158506/0_file_0.zip
2021/04/23 11:56:19 ID:161675 download start:{"frameId":"7B243FE9CAB237BBEBC982C8F9137FB1","guid":"c5a61d9c-c318-41c2-aa4b-bd2f60267a67","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/72e280ce-14e0-4547-b134-9a1d6427352c","suggestedFilename":"0_file_0.zip"}
2021/04/23 11:56:19 ID:161675 download completed:{"guid":"c5a61d9c-c318-41c2-aa4b-bd2f60267a67","totalBytes":1.910861e+06,"receivedBytes":1.910861e+06,"state":"completed"}
2021/04/23 11:56:19 ID:161675, download dir:/app/download/161675 has 1 files: /app/download/161675/0_file_0.zip
2021/04/23 11:56:19 ID:973729 download start:{"frameId":"9F9D4542AA307C35B91ADBB49D8FC750","guid":"86d42975-41a5-44fc-a144-e69ced71024a","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/d85fdc77-c052-48d7-aa5b-e1c8c060d795","suggestedFilename":"0_file_0.zip"}
2021/04/23 11:56:20 ID:973729 download completed:{"guid":"86d42975-41a5-44fc-a144-e69ced71024a","totalBytes":1.910893e+06,"receivedBytes":1.910893e+06,"state":"completed"}
2021/04/23 11:56:20 ID:973729, download dir:/app/download/973729 has 1 files: /app/download/973729/0_file_0.zip
2021/04/23 11:56:20 ID:023296 download start:{"frameId":"169728AEEC8165594E599A46935DB271","guid":"15862b32-130c-407d-a4fc-88ed5635c0f7","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/7074f201-8356-4a99-afca-5a766f0bec85","suggestedFilename":"0_file_0.zip"}
2021/04/23 11:56:20 ID:023296 download completed:{"guid":"15862b32-130c-407d-a4fc-88ed5635c0f7","totalBytes":1.910861e+06,"receivedBytes":1.910861e+06,"state":"completed"}
2021/04/23 11:56:20 ID:023296, download dir:/app/download/023296 has 1 files: /app/download/023296/0_file_0.zip
2021/04/23 11:56:22 ID:673276 download start:{"frameId":"4A1BEB8EF148F4F38336AC0C34A8C173","guid":"6c2bdd25-0560-48ed-b590-b21efb11a185","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/7b17066b-e253-4b2a-a53d-7f7d87079d9d","suggestedFilename":"0_file_0.zip"}
2021/04/23 11:56:22 ID:673276 download completed:{"guid":"6c2bdd25-0560-48ed-b590-b21efb11a185","totalBytes":1.910861e+06,"receivedBytes":1.910861e+06,"state":"completed"}
2021/04/23 11:56:22 ID:673276, download dir:/app/download/673276 has 1 files: /app/download/673276/0_file_0.zip

$ docker exec -it chromedp_sample df -kh /dev/shm                                                                                [0]
Filesystem      Size  Used Avail Use% Mounted on
shm             4.0G     0  4.0G   0% /dev/shm

$ docker logs chromedp_sample
2021/04/23 12:00:58 ID:560509, browser has 1 tabs
2021/04/23 12:00:58 ID:677965, browser has 1 tabs
2021/04/23 12:00:59 ID:579070, browser has 1 tabs
2021/04/23 12:00:59 ID:973601, browser has 1 tabs
2021/04/23 12:00:59 ID:548967, browser has 1 tabs
2021/04/23 12:01:00 ID:498612, browser has 1 tabs
2021/04/23 12:01:00 ID:045884, browser has 1 tabs
2021/04/23 12:01:01 ID:261080, browser has 1 tabs
2021/04/23 12:01:01 ID:347224, browser has 0 tabs
2021/04/23 12:01:52 ID:414201, browser has 1 tabs
2021/04/23 12:01:52 ID:122699, browser has 1 tabs
2021/04/23 12:01:52 ID:041115, browser has 1 tabs
2021/04/23 12:02:03 ID:341667, browser has 1 tabs
2021/04/23 12:02:05 ID:637921, browser has 1 tabs
2021/04/23 12:02:11 ID:717663, browser has 1 tabs
2021/04/23 12:02:14 Failed to lists all the targets in the browser: websocket url timeout reached
2021/04/23 12:02:14 ID:965936, browser has 0 tabs
2021/04/23 12:02:15 Failed to lists all the targets in the browser: websocket url timeout reached
2021/04/23 12:02:18 ID:258136, browser has 0 tabs
2021/04/23 12:02:18 Failed to lists all the targets in the browser: could not dial "ws://<IP_ADDRESS>:39437/devtools/browser/be075c3f-59cb-47ed-97ec-8c853853f61e": context deadline exceeded
2021/04/23 12:02:18 ID:492620, browser has 0 tabs
2021/04/23 12:02:19 ID:548967 received event EventTargetCrashed:{"targetId":"F4521CCEF7741FB1CE618095FD602E85","status":"killed","errorCode":9}
2021/04/23 12:02:19 ID:548967 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:02:24 ID:579070 received event EventTargetCrashed:{"targetId":"0B587C50E6A0EED403D82009FEB6FD96","status":"killed","errorCode":9}
2021/04/23 12:02:24 ID:579070 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:02:49 ID:717663 received event EventConsoleAPICalled:{"type":"error","args":[{"type":"string","value":""}],"executionContextId":2,"timestamp":1619179369428,"stackTrace":{"callFrames":[{"functionName":"","scriptId":"5","url":"http://xxx.s3-website-us-west-2.amazonaws.com/dist/floorplanEditor.99999927653.min.js","lineNumber":1496,"columnNumber":184},{"functionName":"$jscomp.generator.Engine_.nextStep_","scriptId":"5","url":"http://xxx.s3-website-us-west-2.amazonaws.com/dist/floorplanEditor.99999927653.min.js","lineNumber":43,"columnNumber":104},{"functionName":"$jscomp.generator.Engine_.throw_","scriptId":"5","url":"http://xxx.s3-website-us-west-2.amazonaws.com/dist/floorplanEditor.99999927653.min.js","lineNumber":41,"columnNumber":239},{"functionName":"throw","scriptId":"5","url":"http://xxx.s3-website-us-west-2.amazonaws.com/dist/floorplanEditor.99999927653.min.js","lineNumber":44,"columnNumber":114},{"functionName":"V","scriptId":"5","url":"http://xxx.s3-website-us-west-2.amazonaws.com/dist/floorplanEditor.99999927653.min.js","lineNumber":45,"columnNumber":108}]}}
2021/04/23 12:02:50 ID:717663 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: Received unknown error
2021/04/23 12:03:39 ID:347224 received event EventTargetCrashed:{"targetId":"624A71FA38214CAA69E12B4F95DAE34E","status":"killed","errorCode":9}
2021/04/23 12:03:39 ID:347224 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:03:57 ID:677965 received event EventTargetCrashed:{"targetId":"30C03FCC9B4E3FE81D4E5C21F098F50C","status":"killed","errorCode":9}
2021/04/23 12:03:57 ID:677965 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:04:06 ID:973601 received event EventTargetCrashed:{"targetId":"90561779D06BE6B0BAD5109728D5142A","status":"killed","errorCode":9}
2021/04/23 12:04:06 ID:973601 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:04:16 ID:498612 received event EventTargetCrashed:{"targetId":"08C05D05F83A3BDF23C10556AE998B3A","status":"killed","errorCode":9}
2021/04/23 12:04:16 ID:498612 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:05:38 ID:045884 received event EventTargetCrashed:{"targetId":"BCC274CCE66FF1B2A24548C717C76D01","status":"killed","errorCode":9}
2021/04/23 12:05:38 ID:045884 downloaded failed at URL:http://xxx.s3-website-us-west-2.amazonaws.com/?key1=value1&config=autoexport&key2=value2&loc=en&key3=value3: killed
2021/04/23 12:05:48 ID:261080 download start:{"frameId":"9ACF8CFABA591BC0F266735EB93331C6","guid":"e728a024-5ee9-4bdf-bb58-a36948374c58","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/cdd16715-4b6f-47de-a4e7-309b1c7ed5d9","suggestedFilename":"0_file_0.zip"}
2021/04/23 12:05:50 ID:261080 download completed:{"guid":"e728a024-5ee9-4bdf-bb58-a36948374c58","totalBytes":1.910893e+06,"receivedBytes":1.910893e+06,"state":"completed"}
2021/04/23 12:05:50 ID:261080, download dir:/app/download/261080 has 1 files: /app/download/261080/0_file_0.zip
2021/04/23 12:05:54 ID:560509 download start:{"frameId":"2C714F00E1BFF7138FA5EE7CE83BA6EB","guid":"5cd7185a-b4dc-40d0-a8b5-45bdd8a1e6c4","url":"blob:http://xxx.s3-website-us-west-2.amazonaws.com/6f476f0c-0011-4e22-8bfd-8fc8675aa1d6","suggestedFilename":"0_file_0.zip"}
2021/04/23 12:05:54 ID:560509 download completed:{"guid":"5cd7185a-b4dc-40d0-a8b5-45bdd8a1e6c4","totalBytes":1.910861e+06,"receivedBytes":1.910861e+06,"state":"completed"}
2021/04/23 12:05:54 ID:560509, download dir:/app/download/560509 has 1 files: /app/download/560509/0_file_0.zip

I don't know this issue relating to the web page of url that I navigated or something else.

However, is it the good if we do as the above?

It depends ^_^
Most of the time, it will run faster and consume less resources to run a single browser and share it among tabs. But if you need to download multiple files with the same name at the same time, then that could be the way to go. BTW, I found the confusing Browser.setDownloadBehavior today. The doc says:

|allowAndName| allows download and names files according to their download guids.

Maybe you want to give it a try.
For issue 2, try to enable the debug log of the browser and see whether you can find something helpful:
	opts := append(chromedp.DefaultExecAllocatorOptions[:],
		chromedp.Flag("enable-logging", false),
		chromedp.Flag("v", "100"),
		// chrome_debug.log is in $UserDataDir/Defaults if I remember correctly
		chromedp.UserDataDir("provide the user data dir to store debug log"),
	)

W.R.T. Browser.setDownloadBehavior, this is the new command to use in the future. The download commands and events in the Page domain are deprecated.
Please note that the download commands and events in the Browser domain just work with Chrome/Chromium 92.0.4498.0 or later. Check https://github.com/chromedp/examples/blob/master/download_file/main.go for an example. If you can not upgrade the browser, then stick with chromedp v0.6.12. See #807.

@nnguyent Let's dedicate this issue to the race problem of the page.SetDownloadBehavior method. W.R.T.
the chromedp/headless-shell crashed under stress issue, please file a new one if you have more information to provide.
Closing it now.


Offline Maps

Tangram-es does not support natively  offline maps. is there any prototype for this feature

Yup. #982 #1015

Hi, support for this feature is now on master branch. Follow the closed pull requests for usage informations. Documentation will be added soon.


feature-wip remove AggType on unique table with MoW, enable preAggreation, DSIP-018[5/2]

Proposed changes
Issue Number: close #xxx
Problem Summary:
see DSIP-018:https://cwiki.apache.org/confluence/display/DORIS/DSIP-018%3A+Support+Merge-On-Write+implementation+for+UNIQUE+KEY+data+model
This is the patch for stage 5-2
Checklist(Required)

Does it affect the original behavior: (Yes/No/I Don't know)
Has unit tests been added: (Yes/No/No Need)
Has document been added or modified: (Yes/No/No Need)
Does it need to update dependencies: (Yes/No)
Are there any changes that cannot be rolled back: (Yes/No)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

@EmmyMiao87 can you help to review the code change of Planer?
Basically,  when user enabled unique key merge-on-write, we don't need to merge the data with same keys to ensure which row is latest, old row is marked as deleted in DeleteBitmap. So we can enable pre-aggregation to accelerate the query execution, just like processing data in duplicate keys table.


[Bug]: project depends on both chewie ^1.3.5 and share_plus ^7.0.1, version solving failed.

Platform
All
Plugin
share_plus
Version
7.0.1
Flutter SDK
3.7.11
Steps to reproduce
I'm trying to share_plus as dependency to my project but I'm getting the error that you will see in the logs
Code Sample
cupertino_icons: ^1.0.2
  flutter_screenutil: ^5.5.4
  built_value: ^8.0.6
  built_collection: ^5.0.0
  wc_form_validators: ^1.0.0
  cached_network_image: ^3.2.1
  after_layout: ^1.1.0
  dio: ^4.0.6
  intl: ^0.17.0
  bloc: ^7.2.0
  flutter_bloc: ^7.3.3
  hydrated_bloc: ^7.1.0
  animations: ^2.0.3
  page_transition: ^2.0.2
  flutter_i18n: ^0.22.3
  flutter_translate_annotations: ^2.0.0
  dotted_border: ^2.0.0
  flutter_svg: ^0.23.0+1
  shimmer: ^2.0.0
  flutter_spinkit: ^5.1.0
  path_provider: ^2.0.9
  fl_country_code_picker: ^0.0.3+1
  date_time_picker: ^2.1.0
  pin_code_fields: ^7.4.0
  sliver_tools: ^0.2.8
  flutter_slidable: ^2.0.0
  table_calendar: ^3.0.8
  smooth_star_rating_null_safety: ^1.0.4+2
  dotted_line: ^3.1.0
  qr_flutter: ^4.0.0
  chewie: ^1.3.5
  video_player: ^2.4.7
  mobile_scanner: ^2.1.0
  google_maps_flutter: ^2.1.10
  flutter_switch: ^0.3.2
  image_picker: ^0.8.6+1
  carousel_slider: ^4.2.1
  story_view: ^0.14.0
  rich_text_controller: ^1.4.0
  geolocator: ^9.0.1
  uuid: ^3.0.7
  camera: ^0.10.3
  flutter_datetime_picker: ^1.5.1
  video_thumbnail: ^0.5.3
  pusher_channels_flutter: ^2.1.2
  crypto: ^3.0.2
  firebase_core: ^2.8.0
  firebase_messaging: ^14.3.0
  flutter_local_notifications: ^12.0.4
  hyperpay_plugin: ^2.0.0
  url_launcher: ^6.1.10
  permission_handler: ^10.2.0
  agora_rtc_engine: ^6.1.0
  agora_uikit: ^1.3.1
  screenshot: ^1.3.0
  tutorial_coach_mark: ^1.2.8
  shared_preferences: ^2.1.0
  searchfield: ^0.7.7
  pull_to_refresh: ^2.0.0
  share_plus: ^7.0.1

Logs
Because no versions of wakelock_windows match >0.2.0 <0.2.1 and wakelock_windows >=0.2.1 depends on win32 ^3.0.0, wakelock_windows >0.2.0 requires win32 ^3.0.0.
And because wakelock_windows 0.2.0 depends on win32 ^2.0.0, wakelock_windows >=0.2.0 requires win32 ^2.0.0 or ^3.0.0.
And because share_plus >=7.0.1 depends on win32 >=4.0.0 <6.0.0 and wakelock 0.6.2 depends on wakelock_windows ^0.2.0, share_plus >=7.0.1 is incompatible with wakelock 0.6.2.
Because chewie >=1.3.5 depends on wakelock ^0.6.2 and no versions of wakelock match >0.6.2 <0.7.0, chewie >=1.3.5 requires wakelock 0.6.2.
Thus, share_plus >=7.0.1 is incompatible with chewie >=1.3.5.
So, because howwi depends on both chewie ^1.3.5 and share_plus ^7.0.1, version solving failed.
pub get failed

Flutter Doctor
No need

Checklist before submitting a bug

[X] I Google'd a solution and I couldn't find it
[X] I searched on StackOverflow for a solution and I couldn't find it
[X] I read the README.md file of the plugin
[X] I'm using the latest version of the plugin
[X] All dependencies are up to date with flutter pub upgrade
[X] I did a flutter clean
[X] I tried running the example project

Duplicate of https://github.com/fluttercommunity/plus_plugins/issues/1812
It is the problem of wakelock dependency, which is not updated for a long time. Read the referenced issue.
EIther wait till wakelock and chewie are updated or use dependency_overrides.

@vbuberen
Do you have any idea how to use dependency_overrides in that case?

https://letmegooglethat.com/?q=Flutter+dependency+overrides


MTG-Jamendo utility

Summary
This PR adds utility of MTG-Jamendo dataset.
ref: https://mtg.github.io/mtg-jamendo-dataset/

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 94.15584% with 9 lines in your changes missing coverage. Please review.

Project coverage is 82.47%. Comparing base (e1cedbb) to head (dc20c06).

Files
Patch %
Lines

audyn/utils/data/download/__init__.py
76.47%
8 Missing :warning:

audyn/utils/github/__init__.py
66.66%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #191      +/-   ##
==========================================
+ Coverage   82.36%   82.47%   +0.10%
==========================================
  Files         177      180       +3
  Lines       16055    16199     +144
==========================================
+ Hits        13224    13360     +136
- Misses       2831     2839       +8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Release/v0.7.0

Codecov Report

Merging #20 (1cf7ce9) into main (bae54a8) will decrease coverage by 0.28%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##              main      #20      +/-   ##
===========================================
- Coverage   100.00%   99.71%   -0.29%
===========================================
  Files            9        9
  Lines          362      355       -7
  Branches        61       56       -5
===========================================
- Hits           362      354       -8
- Partials         0        1       +1

Impacted Files
Coverage Δ

bods_client/models/siri.py
98.96% <100.00%> (-1.04%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bae54a8...1cf7ce9. Read the comment docs.


Rework expansion areas in UI

Replaced the accordion usages on the form pages with ExpansionSection - seems more appropriate according to patternfly examples.

renamed ConfigureConnectorTypeComponent to ConfigureConnectorComponent
replaced accordion usages with ExpansionSection
indent form inputs to better distinguish the different sections.  Each area is titled, with exansion section below it.

example screenshots
Properties Step:

Data Options Step:

Runtime Options Step:

I think handling expansion with the title itself would be more generic then adding a separate expansion area. Expansion section looks different then actual UX. Maybe we can discuss with UX team.

@uidoyen agree, UX may have some input on this - we can adjust if needed after their review.  It may be better to just remove the titles and use the expansion text as the title.


Update http4s-blaze-client, http4s-circe, ... to 0.21.21

Updates

org.http4s:http4s-blaze-client
org.http4s:http4s-circe
org.http4s:http4s-client
org.http4s:http4s-core
org.http4s:http4s-dsl
org.http4s:http4s-server

from 0.21.20 to 0.21.21.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.http4s" } ]

labels: library-update, semver-patch

Superseded by #138.


Manage the logs

Description

We need a way to deal with the logs file that can end up too big if you leave your instances running. Either delete them after the session is killed, or maybe delete them every hour.
Context

On the simcorp PoV we have a vm that stopped running because the logs took all the space available on the vm.
Other information (if relevant)

I have hundreds of log folders in my ~/.atoti/logs/ folder and I only have a few MB, there might be an issue with your logs if they get so big
Does any lib has such feature of log management so we can see what already exists ? I'm not aware of anything like that for spark

I just checked my logs folder and I've reached 51Gb of logs, I imagine this is due to having larger projects running for long periods of time on my computer. Since things like license checks are continuously logged, if the atoti server is running for a long time it can start to build up.

I also have process running for hours every day with License and health checks logged, I don't think that's the issue.
Can you check if you have a few huge log files that make most of your folder and what's in it ? It could be related to https://github.com/activeviam/atoti/issues/553 for instance, where a log is added for each line of the CSV.

Spark does not automatically clean the logs, but there is an option for it.
See spark.history.fs.cleaner.enabled:
https://spark.apache.org/docs/latest/monitoring.html

create_logging_config() can be used since 0.5.3.


Incorrect search highlight with tab indentation

When searching in a file indented with tabs, the highlight is shifted 7 columns too far to the right (when using 8 space tab width):

VSCode 1.40.2
Neo Vim extension 0.0.39
Neo Vim 0.5.0+ubuntu1+git201912090017-76d1e93-00e710e-795de22~ubuntu18.04.1 (Unstable PPA)

Does the problem goes away if you do :retab command?

No, :retab doesn't seem to have any effect.

Confirmed, thanks for the report. Looks like tab options are not propagated correctly to neovim

I tried the latest master, and the behavior seems significantly worse now (searching for "port"):

Many incorrect pieces of highlighting appear during entering the search string and don't disappear when more characters are typed; even :noh doesn't work.
I've also noticed another related issue: rectangular selections using Ctrl-V aren't rectangular at all when lines indented using a different number of tabs are involved.

Thanks for trying, curious, it works for me (using tabs & tab size - 8). Is it a large file? Setting decorators may be really slow

Also currently it breaks if you change tab size and convert tab2spaces or spaces2tab - need to reopen the file (retab also should help in this case)

Not really large, < 2000 lines. Highlighting doesn't feel slow in any way.
My init.vim only contains set number.

shouldn't matter since i'm overwriting it with set nonumber but try to remove it (you don't need it anyway)

ah and since you're building vsx from master manually, try to remove old extension from ~/.vscode/extensions/asvetliakov.vscode-neovim-0.0.39 before installing master version

Removed my init.vim setting
Removed the old extension directory

This didn't have any effect.
I can't tell at all what triggers the issue, it seems to work for some search strings, and break for others, with no clear pattern. Having a long file with lots of weird indentation (like block comments indented by one space instead of tabs, or mixed tab+space usage for alignment) might have something to do with it, but I'm not sure.
My example file is https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/drivers/tty/serial/mxs-auart.c - searching for AUART seems to work okay at the top of the file, but ASM will show the issue starting around line 135.

well it's strange because this file works for me too

what is your settings for vscode's Editor: detect indentation, Editor: insert spaces and Editor: tab size once again? and the detected tab size in vscode's status bar

Upd: reproduced now, searching for BIT in this screen breaks

spaces between BIT has different tab size

Looks like the problem comes with recently added mutli-byte character support - the cursor position actually is wrong while processing grid_line events so a wrong line is being taken into account for calculating byte offsets 😭

@schiffermtq Could you try with latest master again? Hope this finally fixes it. Thanks!

Thanks, search is working correctly now.
Should I open a separate ticket for the rectangular selection issue with tab indentation?

Yes, please


treewide: depend on shutdown.target if DefaultDependencies=no in almost every case

Description of changes
Made almost all systemd service units throughout the NixOS tree that specify DefaultDependencies=no conflict with with and come before shutdown.target.
@ElvishJerricco:

I mean it makes sense. Before=sysinit.target means it's trying to be part of sysinit, which is almost always the case for services with DefaultDependencies=no. And the ordering and conflict with shutdown.target is just common sense in case a very early shutdown occurs.

shutdown.target

A special target unit that terminates the services on system shutdown.
Services that shall be terminated on system shutdown shall add Conflicts= and Before= dependencies to this unit for their service unit, which is implicitly done when DefaultDependencies=yes is set (the default).)

The few exceptions:

https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/system/boot/kexec.nix
https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/system/boot/shutdown.nix
https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/system/boot/systemd/initrd.nix
https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/system/boot/systemd/shutdown.nix
https://github.com/NixOS/nixpkgs/tree/master/nixos/modules/services/security/haveged.nix

Related
@nikstur's work on #267982.

https://github.com/NixOS/nixpkgs/pull/263203
https://github.com/NixOS/nixpkgs/pull/270727

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[x] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

ping @nikstur @ElvishJerricco :-)

This broke evaluation of the lxdVirtualMachine image jobs.
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImage.aarch64-linux
https://hydra.nixos.org/job/nixos/trunk-combined/nixos.lxdVirtualMachineImage.x86_64-linux
in job ‘nixos.lxdVirtualMachineImage.x86_64-linux’:
error:
       … while evaluating a branch condition

         at /nix/store/0cgy1xrv40pvk2j1d6sjpgczx8qg7ds4-source/nixos/release-combined.nix:17:28:

           16|
           17|   removeMaintainers = set: if builtins.isAttrs set
             |                            ^
           18|     then if (set.type or "") == "derivation"

       … while calling the 'isAttrs' builtin

         at /nix/store/0cgy1xrv40pvk2j1d6sjpgczx8qg7ds4-source/nixos/release-combined.nix:17:31:

           16|
           17|   removeMaintainers = set: if builtins.isAttrs set
             |                               ^
           18|     then if (set.type or "") == "derivation"

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: The option `systemd.services.lxd-agent.unitConfig.Before' has conflicting definition values:
       - In `/nix/store/0cgy1xrv40pvk2j1d6sjpgczx8qg7ds4-source/nixos/modules/system/boot/systemd.nix': "shutdown.target"
       - In `/nix/store/0cgy1xrv40pvk2j1d6sjpgczx8qg7ds4-source/nixos/modules/virtualisation/lxd-agent.nix': [ ]
       Use `lib.mkForce value` or `lib.mkDefault value` to change the priority on any of these definitions.

@mweinelt would you like me to prepare a patch, or are you on it?

Feel free to come up with a patch.


Web service implementation with ESP8266WebServer and ArduinoJson

I have been working on web service to host the sensor readings using ESP8266WebServer and ArduinoJson libraries: code is here.
Basically on call to a server endpoint, the metriful data readings are getting extracted from their respective structs, slightly transformed and written into JsonObject objects. These are then served as json content via ESP8266WebServer.
All works quite okay at first glance, but as sensor is working for a while (e.g. ~2hrs on 3s reading interval), the values for Temperature, Humidity and Air Quality (except the interpretation string) are randomly returned as null.
Is there any specific in the way they are returned from metriful and resp. to be captured into the ArduinoJson objects?

A possible reason is your synchronization with the "ready" signal.
The basic method for the loop() in this program is to wait for the falling edge on the ready signal, then do all the tasks, then repeat.
The output data from the board is only valid for 2.95 seconds (for a 3 second cycle) after the ready signal goes low (see datasheet page 13). If you read during the ready high period, you get bad data.
Here are 3 ways to fix this:

Write the program to guarantee that you complete the loop in < 2.95 seconds
Identify cases where you exceed the time constraint, then discard the data for that loop and just wait for the next ready assertion. E.g. you could read "ready" before and after reading the data, and only use the data if both reads are low.
Use a longer cycle time (100 or 300 seconds) which gives you more time to do tasks.

A possible reason is your synchronization with the "ready" signal.
The basic method for the loop() in this program is to wait for the falling edge on the ready signal, then do all the tasks, then repeat.
The output data from the board is only valid for 2.95 seconds (for a 3 second cycle) after the ready signal goes low (see datasheet page 13). If you read during the ready high period, you get bad data.
Here are 3 ways to fix this:

Write the program to guarantee that you complete the loop in < 2.95 seconds
Identify cases where you exceed the time constraint, then discard the data for that loop and just wait for the next ready assertion. E.g. you could read "ready" before and after reading the data, and only use the data if both reads are low.
Use a longer cycle time (100 or 300 seconds) which gives you more time to do tasks.

I understand what you say, but I believe this is not the case here: in fact the logic I implemented is very similar to the webserver example: in the loop method the client calls are handled in a while loop executed when the ready signal is on false:
  void loop(void) {
  // Wait for the next new data release, indicated by a falling edge on READY
  while (!ready_assertion_event) {
    server.handleClient();
    yield();
  }
  lastRead = millis();
  ready_assertion_event = false;
  // Air data
  ReceiveI2C(I2C_ADDRESS, AIR_DATA_READ, (uint8_t *) &airData, AIR_DATA_BYTES);

When it changes to true, only the ReceiveI2C is called to populate the globally defined structs (e.g. above airData) with the new measurement. There is nothing else happening that would cause the loop to take more than 2.95 sec and resp. invalidate the data. If when in the while loop, a webserver GET call is received, there is another method that pulls data out of the structs, populates into JsonObject and serves back.
The behavior with the nulls described initially is there in all cycle times when running for a while, after a restart it is okay and few hrs later again.
Actually the tests I did show it is related to the use of sprintf formatting function to produce combined string and write into JsonObject. When I change this
  //Humidity
  char H_value[50]={0};
  sprintf(H_value, "%u.%u", airData.H_pc_int, airData.H_pc_fr_1dp);
  JsonObject humidity = doc["Humidity"].createNestedObject();
  humidity["value"] = H_value;
  humidity["unit"] ="%";

to this
//Humidity
  JsonObject humidity = doc["Humidity"].createNestedObject();
  humidity["H_pc_int"] = airData.H_pc_int;
  humidity["H_pc_fr_1dp"] = airData.H_pc_fr_1dp;
  humidity["unit"] ="%";

all works fine - no nulls anymore.
I can definitely work this way and handle the json in the app where it is consumed, but curious about the reason causing it.

I understand what you say, but I believe this is not the case here: in fact the logic I implemented is very similar to the webserver example: in the loop method the client calls are handled in a while loop executed when the ready signal is on false:
  void loop(void) {
  // Wait for the next new data release, indicated by a falling edge on READY
  while (!ready_assertion_event) {
    server.handleClient();
    yield();
  }
  lastRead = millis();
  ready_assertion_event = false;
  // Air data
  ReceiveI2C(I2C_ADDRESS, AIR_DATA_READ, (uint8_t *) &airData, AIR_DATA_BYTES);

When it changes to true, only the ReceiveI2C is called to populate the globally defined structs (e.g. above airData) with the new measurement. There is nothing else happening that would cause the loop to take more than 2.95 sec and resp. invalidate the data. If when in the while loop, a webserver GET call is received, there is another method that pulls data out of the structs, populates into JsonObject and serves back.
The behavior with the nulls described initially is there in all cycle times when running for a while, after a restart it is okay and few hrs later again.
Actually the tests I did show it is related to the use of sprintf formatting function to produce combined string and write into JsonObject. When I change this
  //Humidity
  char H_value[50]={0};
  sprintf(H_value, "%u.%u", airData.H_pc_int, airData.H_pc_fr_1dp);
  JsonObject humidity = doc["Humidity"].createNestedObject();
  humidity["value"] = H_value;
  humidity["unit"] ="%";

to this
//Humidity
  JsonObject humidity = doc["Humidity"].createNestedObject();
  humidity["H_pc_int"] = airData.H_pc_int;
  humidity["H_pc_fr_1dp"] = airData.H_pc_fr_1dp;
  humidity["unit"] ="%";

all works fine - no nulls anymore.
I can definitely work this way and handle the json in the app where it is consumed, but curious about the reason causing it.

It looks like a memory problem relating to ArduinoJson (which we are not able to help with).
You can check whether it is a sprintf problem by including the sprintf line, printing the resulting H_value, and removing the parts with the JsonObjects.
This line:
humidity["value"] = H_value;
is presumably copying the string into the json object. Can it run out of memory if you keep doing this? Maybe call memoryUsage() on the DynamicJsonDocument to check, or just clear() it after every send?
Our examples avoid dynamic memory allocation because of these difficulties!
In any case, it seems the problem is not with our hardware/software.

It looks like a memory problem relating to ArduinoJson (which we are not able to help with).
You can check whether it is a sprintf problem by including the sprintf line, printing the resulting H_value, and removing the parts with the JsonObjects.
This line:
humidity["value"] = H_value;
is presumably copying the string into the json object. Can it run out of memory if you keep doing this? Maybe call memoryUsage() on the DynamicJsonDocument to check, or just clear() it after every send?
Our examples avoid dynamic memory allocation because of these difficulties!
In any case, it seems the problem is not with our hardware/software.

Thanks a lot for your help here. Will keep looking into it and post an update if find the exact cause.
Nevertheless fully agree this now is clearly not at all related to the Metriful product or software!

Thanks a lot for your help here. Will keep looking into it and post an update if find the exact cause.
Nevertheless fully agree this now is clearly not at all related to the Metriful product or software!


Mediator appears to double-post

NOTE: Let's double-check, but I have a hunch this is because the LLM response to both participants A and B are the same (e.g., participant A posts, LLM response A loading, participant B posts, LLM response B loading... but since posting messages takes a while, both responses A and B post).


Proofread VRX 2023 Documentation

We now have version 1.1 of both our technical guide and task descriptions for VRX 2023. Before our final release, it would be good for someone to proofread both of these documents. Creating this issue here so we can track it with the milestone.

I volunteer  :+1:

@caguero Apologies, this issue should have specified that it is for version 1.2 of the documents, since version 1.1 is already published. I will transfer your comments over. Just clarifying here also in case there are further changes / comments from others.

@M1chaelM Could you put a link to the latest version in the meeting agenda so I can be sure I'm reviewing the right copy with any new comments?

Reconfirming here - reviewed both docs, see comments for recommendations

@caguero and @j-herman Most comments are now resolved. Please take a quick look at the outstanding comment threads.

All comments resolved. Version 1.2 of both documents is ready for publication. No further changes to the rules, API or environmental envelopes is expected.

oops, reopening to address https://github.com/osrf/vrx/pull/692


Unable to generate token

Description
I get an error whenever I call the getToken function. The error is:
[getToken] Error/Exception: {"nativeStackAndroid":[],"userInfo":null,"message":"6003: certificate fingerprint error","code":"907122045","line":2242,"column":45,"sourceURL":"http://<IP_ADDRESS>:8081/index.bundle?platform=android&dev=true&minify=false"}
I already added enabled the push kit, added the SHA-256 certificate fingerprint, added the agconnect-services.json inside my android/app folder. Setup my gradle:
buildscript {
    ext {
        buildToolsVersion = "29.0.2"
        minSdkVersion = 21
        compileSdkVersion = 29
        targetSdkVersion = 29
    }
    repositories {
        google()
        jcenter()
        maven {url 'https://plugins.gradle.org/m2/'}
        maven {url 'https://developer.huawei.com/repo/'}
    }
    dependencies {
        classpath("com.android.tools.build:gradle:3.5.3")
        classpath 'gradle.plugin.com.onesignal:onesignal-gradle-plugin:[0.12.9, 0.99.99]'
        classpath 'com.huawei.agconnect:agcp:<IP_ADDRESS>1'
        // NOTE: Do not place your application dependencies here; they belong
        // in the individual module build.gradle files
    }
}

allprojects {
    repositories {
        mavenLocal()
        maven {
            // All of React Native (JS, Obj-C sources, Android binaries) is installed from npm
            url("$rootDir/../node_modules/react-native/android")
        }
        maven {
            // Android JSC is installed from npm
            url("$rootDir/../node_modules/jsc-android/dist")
        }

        google()
        jcenter()
        maven { url 'https://www.jitpack.io' }
        maven {url 'https://developer.huawei.com/repo/'}
    }
}

and added
implementation 'com.huawei.hms:push:<IP_ADDRESS>0'
implementation project(':react-native-hms-push')
Not sure what went wrong based on the error it generated.
Expected behavior
It should return a token
Current behavior
returns an error:
[getToken] Error/Exception: {"nativeStackAndroid":[],"userInfo":null,"message":"6003: certificate fingerprint error","code":"907122045","line":2242,"column":45,"sourceURL":"http://<IP_ADDRESS>:8081/index.bundle?platform=android&dev=true&minify=false"}
Screenshots
If applicable, add screenshots to help explain your issue
Environment
"@hmscore/react-native-hms-push": "^5.0.2-301",
"react": "16.13.1",
"react-native": "0.63.4",

Hi,
The error code 6003 indicates that there's an issue with keystore file configurations. You may see the error code table here. For the error you've encountered, please follow the instructions in the "Suggestion" column.

Hi, I also encounter this issue lately.

Hi, I encounter this issue when I try to call getToken method using the cloud debugging.
[getToken] Error/Exception: {"nativeStackAndroid":[], "userinfo":null, "message":"907135000: arguments invalid", "code": "907122045", "line": 18, "column": 1111, "sourceURL":"index.android.bundle"}

And this when I using my device (Realme 6)
Note that I have check using react-native-device-info wheteher this device support hms. It return true so Im assuming that we should able to get the token.
[getToken] Error/Exception: {"nativeStackAndroid":[],"userInfo":null,"message":"907135003: client api invalid","code":"907122045"}

I have follow all the development guide provided in the huawei developer on how to integrate HMS Core Push Kit in react native. Just that I need to uncomment this line in MainApplication.java due to autolinking.
// packages.add(new HmsPushPackage());
I have check here for the error code description
"907135000" - HmsPushResultCode.ERROR_INTERNAL_ERROR - Internal error. Contact Huawei technical support.
"907122045" - HmsPushResultCode.ERROR_UNKNOWN - Unknown error. Contact Huawei technical support.

Any idea what going wrong here? I get a response calling getId, getAAID, getCreationTIme, enable/disable autoInit and IsAutoInitEnabled method.

Hi,
The error message 907135000 means the SDK is misconfigured, please check the instructions here and compare with your own configurations. As for 907135003, check if your device has an access to internet.

Hi ozan,

Hi,
The error message 907135000 means the SDK is misconfigured, please check the instructions here and compare with your own configurations. As for 907135003, check if your device has an access to internet.

Hi,
May I know which error code message should I referred to? As I try to implement this in react native, which error code message should I refer to? Is it from react-native guide or android guide

Anyways, It turned out that the reason why I am unable to retrieve the token is because I download the HMS Core from play store. Error prompted showing that unable to open HMS Core.

@neilakoh you need add ### apply plugin: "com.huawei.agconnect" in app/build.gradle


Add amm network

High Level Overview of Change

Add amm network to VHS.
Context of Change

Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] Refactor (non-breaking change that only restructures code)
[ ] Tests (You added tests for code that already exists, or your new feature included in this PR)
[ ] Documentation Updates
[ ] Release

Before / After

Test Plan

To add the AMM network, you only need the changes to network.ts. What's the purpose of all the other changes?

Add inline comments to explain each of them. Please let me know if you have questions.


Add unFederalReserve and UN tokens

Hi guys,
We would like to add unFederalReserve and our UN ERC-20 tokens to MetaMask.
Website: https://unfederalreserve.com/
Etherscan: https://etherscan.io/token/0x5218e472cfcfe0b64a064f055b43b4cdc9efd3a6
CoinGecko: https://www.coingecko.com/en/coins/unfederalreserve
GitHub: https://github.com/UnFederalReserve
Twitter: https://twitter.com/Unfederalreser1
Medium: https://unfederalreserve.medium.com/
ReserveLending: https://lending.unfederalreserve.com/
TokenList.org: https://tokenlists.org/token-list?url=https://raw.githubusercontent.com/UnFederalReserve/token-lists/main/unfederalreserve_tokenlist.json
Description
unFederalReserve is a banking SaaS company built on blockchain technology. Our banking products are designed for smaller U.S. Treasury chartered banks and non-bank lenders in need of greater liquidity without sacrificing security or compliance.
Reserve Lending on the unFederalReserve platform is a fork of Compound. For everyday DeFi users, this means the Reserve Lending platform benefits from a long history of stable, secure transactions. As a B2B lending platform, unFederalReserve provides tools to businesses that do not have access to more traditional methods of borrowing.

Please change addresses to EIP-55 version

Updated unFederalReserve addresses to EIP-55 version.  Please review changes for submission.

@xrn I see the check failed but not seeing what tokens it mentioned it is overlapping
# symbols should not overlap
ok 4904 symbols should not overlap

1..4904
# tests 4904
# pass  4903
# fail  1

Error: Process completed with exit code 1.

@xrn let me know if there is anything on my end that is needed to get this to pass checks and get properly merged.  Thanks again for all the help here!

@b1ckchn can you please merge our master to your code and add to this PR - I made changes in tests which allows for symbols longer than 6 chars

@b1ckchn can you please merge our master to your code and add to this PR - I made changes in tests which allows for symbols longer than 6 chars

@xrn Thanks!  merged your master into local branch which updated code.  See if things look better now.


remove last separator

For Issue: https://github.com/tmc/langchaingo/issues/285
PR Checklist

[x] Read the Contributing documentation.
[x] Read the Code of conduct documentation.
[x] Name your Pull Request title clearly, concisely, and prefixed with the name of the primarily affected package you changed according to Good commit messages (such as memory: add interfaces for X, Y or util: add whizzbang helpers).
[x] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. Fixes #123).
[x] Describes the source of new concepts.
[x] References existing implementations as appropriate.
[x] Contains test coverage for new functions.
[x] Passes all golangci-lint checks.

We can also use the template function "trimSuffix" to remove the last suffix which provided by sprig PACKAGE. For example:
{{ .context | trimSuffix "\n" }}


Finish the Markdown reference/tutorial

[x] markdown home page
[x] extensions page (reference)
[ ] some sort of tutorial, but where/in what section? (see message below)

#62 added a stub page for basic Markdown syntax and we need to complete it. The page should be minimal and easy to scan. Any nonstandard Markdown features were extracted into a separate page.
Basic Markdown that most users use is extremely simple and should focus on that. For example, GitHub's Markdown Cheat Sheet looks like this:

Include the syntax for:

Emphasis
Code Block
Lists
Links
Blockquotes

I'm not sure if it's worth including the result. For most of them, it'll look very similar to the original syntax and it's pretty obvious. It can look slightly different on Codewars because different libraries are used and CSS is different. For Headers, it'll also mess with the table of contents as well.

about the math part: https://github.com/codewars/codewars.com/issues/1527

I'm planning on doing the extensions page as a reference, so I'll prune it from some tiny parts (for now I'll keep them commented out for later).
Thing is, I'd like to create a tutorial about markdown, but I'm not sure how to tackle it, since it can be usefull for creating kata (description) as well as for comments.
Hence, simpler course of action should be, I guess, to make it a stand alone thing, but I feel it a bit weird at the same time. Any thoughts? (Maybe I'll just give it a shot in some time and we'll see after that?)

Up here! To see if we close this one or not.

Do you (plural) think we need a tutorial or recipe about markdowns yet?
If so, do we make it a stand alone thing, or do we rather incorporate it to something else (like, writing descriptions)?

Not really for Markdown itself, but I think we should have something for "how to write comments effectively" and also "how to write kata descriptions". For kata description, we should have some recommended style (what to include, how to structure, etc).

ok, I'll open a temporary "todo" issue to kee track of this.
:+1:


[android] Fix IRGen/condfail.sil test for Android ARMv7/AArch64.

For AArch64, the fix is duplicating the arm64 lines, since those are not
taken when testing Android/Linux which use AArch64 as the architecture.
For ARMv7, the change is supporting both trap and .inst 0xe7ffdefe.
According to llvm/lib/Target/ARM/ARMAsmPrinter.cpp, non-Darwin binutils
do not support the mnemonic trap, so an .inst is emitted instead. The
same instruction has to be used in both places, though.
For both architectures add the Android version of APP/NO_APP, which uses
@-symbols instead of ##.

Modified to have different APP/NO_APP checks for Android ARMv7 and AArch64. This fix that the checks were not actually checked at all (I haven’t fixed the Windows ones, but I’m quite sure that they are not triggering, btw).
Also remove the variable for the trap instruction. It doesn’t give us a lot more safety, and x86_64 does something similar with .cfi_startproc|Lfunc_begin

@swift-ci please test and merge


Atlys - Extra /dev/videoX

/dev/video0 good, video1 .. bad?
power on Atlys:
[  151.146340] usb 1-1.3.1: new high-speed USB device number 8 using dwc_otg
[  151.377038] usb 1-1.3.1: config 1 interface 0 altsetting 0 bulk endpoint 0x1 has invalid maxpacket 64
[  151.377064] usb 1-1.3.1: config 1 interface 0 altsetting 0 bulk endpoint 0x81 has invalid maxpacket 64
[  151.378066] usb 1-1.3.1: string descriptor 0 read error: -32
[  151.378096] usb 1-1.3.1: New USB device found, idVendor=1443, idProduct=0007, bcdDevice= 0.00
[  151.378108] usb 1-1.3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[  151.776373] usb 1-1.3.4: new full-speed USB device number 9 using dwc_otg
[  152.009207] usb 1-1.3.4: New USB device found, idVendor=04e2, idProduct=1410, bcdDevice= 0.03
[  152.009231] usb 1-1.3.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[  152.029606] cdc_acm 1-1.3.4:1.0: ttyACM1: USB ACM device
[  152.517499] vizzini: loading out-of-tree module taints kernel.
[  152.531199] vizzini 1-1.3.4:1.0: This device cannot do calls on its own. It is not a modem.
[  152.531333] vizzini 1-1.3.4:1.0: ttyVIZ0: XR21v14x usb uart device
[  152.558161] usbcore: registered new interface driver vizzini
[  152.558178] vizzini: Exar USB UART Driver for XR21V141x

pi@oppi:~/prebuilt $ ~/HDMI2USB-mode-switch/venv/bin/hdmi2usb-mode-switch -v --by-type atlys --load-fx2-firmware hdmi2usb.hex
[  342.751228] usb 1-1.3.1: USB disconnect, device number 8
wut?
[  344.426418] usb 1-1.3.1: new high-speed USB device number 10 using dwc_otg
[  344.657205] usb 1-1.3.1: config 1 interface 2 altsetting 0 endpoint 0x81 has an invalid bInterval 64, changing to 10
[  344.658037] usb 1-1.3.1: New USB device found, idVendor=1d50, idProduct=60b7, bcdDevice= 0.02
[  344.658052] usb 1-1.3.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  344.658063] usb 1-1.3.1: Product: HDMI2USB.tv - Numato Opsis Board
[  344.658071] usb 1-1.3.1: Manufacturer: TimVideos.us
[  344.658080] usb 1-1.3.1: SerialNumber:<PHONE_NUMBER>abcdef
[  344.690326] cdc_acm 1-1.3.1:1.2: ttyACM1: USB ACM device
[  345.147841] uvcvideo: Found UVC 1.00 device HDMI2USB.tv - Numato Opsis Board (1d50:60b7)
[  345.148316] uvcvideo: UVC non compliance - GET_DEF(PROBE) not supported. Enabling workaround.
[  345.150919] uvcvideo 1-1.3.1:1.0: Entity type for entity Extension 3 was not initialized!
[  345.150945] uvcvideo 1-1.3.1:1.0: Entity type for entity Processing 2 was not initialized!
[  345.150959] uvcvideo 1-1.3.1:1.0: Entity type for entity Camera 1 was not initialized!
[  345.154216] usbcore: registered new interface driver uvcvideo
[  345.154230] USB Video Class driver (1.1.1)

pi@oppi:~/prebuilt $ tree /dev/hdmi2usb
/dev/hdmi2usb
├── by-num
│   ├── all0
│   │   ├── tty -> ../../../ttyACM1
│   │   ├── usbdev -> ../../../bus/usb/001/010
│   │   └── video -> ../../../video1
│   └── atlys0
│       ├── tty -> ../../../ttyACM1
│       ├── usbdev -> ../../../bus/usb/001/010
│       └── video -> ../../../video1
├── by-path
│   ├── tty -> ../../../ttyACM1
│   ├── usb.bus1.port1-hub.port3-hub.port1
│   │   ├── tty -> ../../../ttyACM1
│   │   ├── usbdev -> ../../../bus/usb/001/010
│   │   └── video -> ../../../video1
│   ├── usbdev -> ../../../bus/usb/001/010
│   └── video -> ../../../video1
└── by-serial
├──<PHONE_NUMBER>abcdef
│   └── usbdev -> ../../../bus/usb/001/010
├── tty -> ../../../ttyACM1
└── video -> ../../../video1
pi@oppi:~/litex-buildenv-jenkins/hdmi2usb/tests $ gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=3 ! jpegdec ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.258886279
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

pi@oppi:~/litex-buildenv-jenkins/hdmi2usb/tests $ gst-launch-1.0 v4l2src device=/dev/video1 num-buffers=3 ! jpegdec ! fakesink
Setting pipeline to PAUSED ...
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video1' is not a capture device.
Additional debug info:
v4l2_calls.c(633): gst_v4l2_open (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Capabilities: 0x4a00000
Setting pipeline to NULL ...
Freeing pipeline ...

FYI - @CarlFK It is expected that the device disconnects and reconnects every time you use hdmi2usb-mode-switch tool

Please reopen if you reproduce this with the udev rules from this repository.


added test helper support for FileInfoBase.Replace

Um, since when should a file system "ShouldCreateBackup".  That's not
normally a filesystem goal, except in very specific instances.
On Sat, Jul 16, 2016 at 8:53 AM, Volodymyr Dombrovskyi <
<EMAIL_ADDRESS>wrote:

You can view, comment on, or merge this pull request online at:
https://github.com/tathamoddie/System.IO.Abstractions/pull/164
Commit Summary

added test helper support for FileInfoBase.Replace

File Changes

M TestHelpers.Tests/MockFileInfoTests.cs
https://github.com/tathamoddie/System.IO.Abstractions/pull/164/files#diff-0
(71)
M TestingHelpers/MockFileInfo.cs
https://github.com/tathamoddie/System.IO.Abstractions/pull/164/files#diff-1
(25)

Patch Links:

https://github.com/tathamoddie/System.IO.Abstractions/pull/164.patch
https://github.com/tathamoddie/System.IO.Abstractions/pull/164.diff

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tathamoddie/System.IO.Abstractions/pull/164, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACG_zJ7ERQ7G8KuUbK3awG45WjxXyGT8ks5qWNRQgaJpZM4JOAa5
.

What about this PR needs to be changed to get it merged in?
According to the docs for System.IO.File.Replace, it looks like a backup is part of the functionality.
You can see in the reference source, it also mentions a backup path, although the actual operation is in native code.
I need this functionality to get my tests working again and would appreciate if it was in the next release.

@rkoeninger we're starting to get through all of the open PRs after some inactivity.
Given that the original date on this PR is quite old, and the original author does not seem to be as active as you on GitHub -- if you would like that champion this enhancement, I'm OK with closing this out and reviewing yours if you still have the desire to do so.

Thanks! I'll close this one and take a look.


Java image is broken

The image for Java does not show up and throws 403 error.

Please assign this to me

sure , /assign @mwasifk

I have submitted a pull request

I am worried about copyright claims and whatnot; please assign someone else to this issue.


Angular debug tests only works first time then breakpoint ignored and then Another test operation is still running error occurs

I have a problem whereby after debugging a test once the breakpoints stop working for the next run and then the subsequent test run fails because.
I have to restart VS code to be able to debug again.
I have 'Test Explorer: Use Native Testing' enabled (but the issue happens without native testing too) and its set to use https and not the cli.
Steps to Reproduce the Behavior

Open VS Code and wait for tests to load
Add a breakpoint on a single test and debug it (works fine)
Once the test is complete try to debug the test again
Test completes but breakpoint is ignored (even though it is shown as attached)
Try to debug the test again
"New test debug request ignored - Another test operation is still running" error occurs

Describe the Expected Behavior

I can repeatedly debug tests without restarting VS code.

I have attached my logs from Test Explorer and Server
Please Provide the Following Information

OS and Version: Windows 11
Environment: WSL2 (Ubuntu)
Node Version: v18.15.0
VS Code Version: 1.84.2
Karma Version: 6.4.0
Browser and Version: Chrome headless
Frameworks: Jasmine
Angular Version (if using Angular): Angular 15

This has always been a bit of an intermittent problem in the past but it is a lot worse now and can be reproduced every time with the steps above.
karma log.txt
karma server log.txt
I suspect the issue is related to my config but any help would be greatly appreciated!

@slagbat Note that with the deprecation of Karma runner, this extension is also now deprecated. While you continue to use Karma and this extension, I encourage you to post any new issues in the Discussions area to possibly get help from other users.
Having said that, I see this is on WSL, and seems very related to issue #88. Perhaps take a look at the discussion in that issue in case you find it helpful in resolving this issue. If not, post in the Discussions area in case other users are able to help.


http: Template view engine support should be added

Template engines can be used using the @istanbul/view plugin. Currently the default pug is used and the only tested engine is pug. Other engines will be tested in the future.
see this issue in view repo for template engine test list


Fix some rescue calls that do not specifiy error type.

What was the end-user problem that led to this PR?
The problem I noticed was in style, several instances of rescue clauses that do not specify an exception type. This is noted in in the Ruby Style Guide as Avoid rescuing the Exception class.
What was your diagnosis of the problem?
My diagnosis was that at least some of those are leftover style. They are noted in .rubocop_todo.yml. To make sure, I asked on Slack.
What is your fix for the problem, implemented in this PR?
My fix is to make as many of them more specific, specifying at least StandardError, and trying to be more specific.
Why did you choose this fix out of the possible options?
No other ways of addressing this came to mind. I'd be happy to consider.

Hi, please fill out the PR template.

How did you decide between Runtime and Standard error?
I preferred RuntimeError as it is more specific than StandardError, won't rescue a SystemCallErrorfor example. When usingRuntimeErrorcaused a failure of the tests and I was not certain which narrowerExceptionmay need rescuing, I resorted toStandardError`. Perhaps those cases are redundant? Would you have them removed?

This looks good to me once the tests are green. 👍

ping @utilum

Sorry for the delay.
I don't have Ruby 1.8.7 locally, and that's where the test failures occur. Just looking at the test log I wonder if the fix to this would not actually be in updating the tests... Anyone?

Anyone?

@bundlerbot r+


Experimental: defining "extension" slot as a list of key-value pairs

Resolves [#ISSUE, #ISSUE]

[ ] docs/ have been added/updated if necessary
[ ] make test has been run locally
[ ] tests have been added/updated (if applicable)
[ ] CHANGELOG.md has been updated.

[Description, mentioning at least relevant #ISSUE and how it was addressed. A bulleted list of all changes performed by the PR is is helpful.]

Superceded by #375


"bigchaindb start" sometimes gets bigchaindb.common.exceptions.InvalidHash and hangs

BigchainDB version: 1.3.0 (released version), and the current master branch of bigchaindb/bigchaindb.
Backend DB: MongoDB
OS: Ubuntu 16.04 remote and Ubuntu 17.10 local (and maybe others)

This issue was first reported by @charlespetchsy on Gitter; see https://gitter.im/bigchaindb/bigchaindb?at=5a14566c232e79134db5100a
I was able to reproduce it on my local machine (running Ubuntu 17.10), with both the current master branch and the v1.3 release branch (which was just released).
Sample Output
(bdb) troy@troy-thinkpad:~/temp-bdb$ rm ~/.bigchaindb
(bdb) troy@troy-thinkpad:~/temp-bdb$ bigchaindb --version
bigchaindb 1.3.0
(bdb) troy@troy-thinkpad:~/temp-bdb$ bigchaindb -y configure mongodb
Generating keypair
Generating default configuration for backend mongodb
Configuration written to /home/troy/.bigchaindb
Ready to go!
(bdb) troy@troy-thinkpad:~/temp-bdb$ bigchaindb start
INFO:bigchaindb.config_utils:Configuration loaded from `/home/troy/.bigchaindb`
[2017-11-21 20:41:19] [INFO] (bigchaindb.commands.bigchaindb) BigchainDB Version 1.3.0 (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.commands.bigchaindb) Initializing database (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.backend.mongodb.connection) Closing initial connection to MongoDB (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.commands.bigchaindb) Starting BigchainDB main process with public key 81fUUBTmDJbqsaM7NZGExUcBswqKQ6z1nYPpzLtQXgX1 (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.processes) Initializing BigchainDB... (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.processes) Starting block (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.backend.mongodb.connection) Closing initial connection to MongoDB (Process-10 - pid: 27077)
[2017-11-21 20:41:19] [INFO] (bigchaindb.processes) Starting voter (MainProcess - pid: 27063)
[2017-11-21 20:41:19] [INFO] (bigchaindb.backend.mongodb.connection) Closing initial connection to MongoDB (Process-10 - pid: 27077)
[2017-11-21 20:41:19] [INFO] (bigchaindb.backend.mongodb.connection) Closing initial connection to MongoDB (MainProcess - pid: 27063)
Traceback (most recent call last):
  File "/home/troy/temp-bdb/bdb/bin/bigchaindb", line 11, in <module>
    sys.exit(main())
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/commands/bigchaindb.py", line 356, in main
    utils.start(create_parser(), sys.argv[1:], globals())
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/commands/utils.py", line 206, in start
    return func(args)
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/commands/utils.py", line 51, in configure
    command(args)
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/commands/utils.py", line 78, in start_logging
    command(args)
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/commands/bigchaindb.py", line 209, in run_start
    processes.start()
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/processes.py", line 57, in start
    vote.start()
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/pipelines/vote.py", line 183, in start
    pipeline = create_pipeline()
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/pipelines/vote.py", line 160, in create_pipeline
    voter = Vote()
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/pipelines/vote.py", line 39, in __init__
    self.last_voted_id = Bigchain().get_last_voted_block().id
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/core.py", line 608, in get_last_voted_block
    return Block.from_dict(self.get_block(last_block_id))
  File "/home/troy/temp-bdb/bdb/lib/python3.6/site-packages/bigchaindb/models.py", line 303, in from_dict
    raise InvalidHash()
bigchaindb.common.exceptions.InvalidHash

At this point it hangs (i.e. I don't get a prompt). I can un-hang it with CTRL + C

I found that I was able to get BigchainDB Server to run normally by shutting down MongoDB (sudo mongod --shutdown) then completely wiping out the /data/db directory (i.e. sudo rm -rf /data/db) and then recreating the /data/db directory with the correct permissions as per the instructions in https://docs.bigchaindb.com/projects/server/en/master/quickstart.html ).
I suppose a more graceful way to wipe the database would be to use bigchaindb drop but I forgot about that (and so I didn't try it).
I was thinking about why this might be happening and now I think I know:  when we merged pull request #1812 (metadata full text search) on Monday (November 20), that added a new table to the database for storing metadata (and indexing it for full text search). Metadata is now removed from the transactions in a block before storing the block in the bigchain table. If you had data in your MongoDB database from before the merge, and then you tried to run BigchainDB again after the merge, then problems arose. Wiping the MongoDB database stops the problems.