Download via storage::Client::ReadObject() does not finish, error, or progress.

A customer reports that sometimes the downloads fail to make progress, with the following traces:
$ strace -ttt -p 28864 2>&1 | head -n 30
Process 28864 attached
1560630449.797041 restart_syscall(<... resuming interrupted call ...>) = 0
1560630449.797242 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.798477 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.798597 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.799807 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.801022 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.801112 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.802329 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.803598 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.803801 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.805009 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.806264 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.806470 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.807739 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.809041 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.809141 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.810410 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0
1560630449.811640 poll([{fd=22, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
1560630449.811854 poll([{fd=22, events=POLLIN}], 1, 1) = 0 (Timeout)
1560630449.813107 nanosleep({0, 1000000}, 0x7ffc2bc087c0) = 0

$ ls -al /proc//28864/fd/22
[snip] /proc//28864/fd/22 -> socket:[213043]

$ lsof | grep 213043
snip...   22u     IPv4 213043       0t0       TCP [censored customer machine]-><IP_ADDRESS>:https (ESTABLISHED)

#0  0x00002b11459cc00d in nanosleep () from /lib64/libpthread.so.0
#1  0x00002b116f54440b in std::__1::this_thread::sleep_for(std::__1::chrono::duration<long long, std::__1::ratio<1l,<PHONE_NUMBER>l> > const&) () from [censored]/lib/libc++.so.1
#2  0x00002b116f379442 in void std::__1::this_thread::sleep_for<long long, std::__1::ratio<1l, 1000l> >(std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> > const&) ()
  from [censored]/lib/libstorage_client.so.1
#3  0x00002b116f376036 in google::cloud::storage::v1::internal::CurlDownloadRequest::WaitForHandles(int&) () from [censored]/lib/libstorage_client.so.1
#4  0x00002b116f3747a0 in google::cloud::v0::Status google::cloud::storage::v1::internal::CurlDownloadRequest::Wait<google::cloud::storage::v1::internal::CurlDownloadRequest::Read(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)::$_1>(g
oogle::cloud::storage::v1::internal::CurlDownloadRequest::Read(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&)::$_1&&) () from [censored]/lib/libstorage_client.so.1
#5  0x00002b116f373a3d in google::cloud::storage::v1::internal::CurlDownloadRequest::Read(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) ()
  from [censored]/lib/libstorage_client.so.1
#6  0x00002b116f439c1d in google::cloud::storage::v1::internal::RetryObjectReadSource::Read(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) ()
  from [censored]/lib/libstorage_client.so.1
#7  0x00002b116f4122dd in google::cloud::storage::v1::internal::ObjectReadStreambuf::Peek() () from [censored]/lib/libstorage_client.so.1
#8  0x00002b116f4127c4 in google::cloud::storage::v1::internal::ObjectReadStreambuf::underflow() () from [censored]/lib/libstorage_client.so.1
#9  0x00002b116f4f301a in std::__1::basic_streambuf<char, std::__1::char_traits<char> >::uflow() () from [censored]/lib/libc++.so.1
#10 0x00002b116f4f2fca in std::__1::basic_streambuf<char, std::__1::char_traits<char> >::xsgetn(char*, long) () from [censored]/lib/libc++.so.1
#11 0x00002b116f4f612d in std::__1::basic_istream<char, std::__1::char_traits<char> >::read(char*, long) () from [censored]/lib/libc++.so.1

Just updating the bug with what I have discovered so far:

I can reproduce the bug using the program at:
https://github.com/coryan/google-cloud-cpp/tree/read-object-stall-regression/google/cloud/storage/tests/read_object_stall_regression

I need to run 250 copies of the program, on about 35 machines to repro the bug. These are not meant as minimal numbers, it is possible that 100 copies will repro the bug too, but it seems that small numbers of copies does not do it.

When the bug presents itself the test program terminates with a message saying "[FATAL] Wait()," and then a lot of logs showing the local and remote endpoint of the failed CURL handle, the headers received on that handle, the contents of /proc/self/net/tcp for the socket associated with that handle.

The Kubernetes job also runs a program to capture the tcpdump of the relevant traffic, and strace to capture the system calls. I have attached (filtered) versions of these captures for one run.

repro.tar.gz

My observations on the attached data are below. I do not know what these means other than "I do not understand this data yet":

Looking at the tcpdump, communication is quite normal until about packet 1027 (in the very filtered output), when the client closes the window:

From that point, the client never opens the window by more than 5632 bytes, the server sends periodic ACKs, but the client only acks very slowly:

Meanwhile the application is just calling poll(). Unfortunately there are no timestamps in the strace log because using strace -tt makes the problem not reproducible :angry:

 118124 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118125 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118126 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118127 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118128 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118129 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118130 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118131 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118132 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118133 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118134 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118135 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118136 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118137 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118138 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118139 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118140 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118141 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118142 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118143 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118144 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 118145 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 118146 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)

Note that the last read() returned EAGAIN:

  95497 poll([{fd=7, events=POLLIN}], 1, 10)    = 1 ([{fd=7, revents=POLLIN}])
  95498 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=7, revents=POLLIN|POLLRDNORM}])
  95499 read(7, "\27\3\3 \21", 5)               = 5
  95500 read(7, "1)d\275>fu\326\267\"\241\f\325\223q\363\240\3\223\373\265\342F\222\342\t\350U\vt\r_"..., 8209) = 7609
  95501 read(7, 0x5566d3915ee1, 600)            = -1 EAGAIN (Resource temporarily unavailable)
  95502 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
  95503 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)

Also note that the socket queues are reported as empty by /proc/self/net/tcp:

147 [FATAL] Wait(), buffer_size_=1048576, buffer_offset_=655360, spill_.size()=16384, spill_offset_=0, closing=0, closed=0, paused=0, in_multi=1, repeats=11313, repeats_deadline=0 too many repeats, exit with    147  error ***###*** (/w/google/cloud/storage/internal/curl_download_request.cc:214)
148 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:125] google::cloud::storage::v1::internal::CurlDownloadRequest::Wait(Predicate)::<lambda()> [with Predicate = google::cloud::storage::v1::i    148 nternal::CurlDownloadRequest::Read(char*, std::size_t)::<lambda()>](0x5566d178f090)   =============================================
149 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:127] (0x5566d178f090)  url_=https://storage-download.googleapis.com/coryan-test-us-west1/ob-Pg7JNTdIeburqcPt.txt, user_agent=gcloud-cpp/v1.    149 3.0+ libcurl/7.64.0-DEV OpenSSL/1.1.1 zlib/1.2.11 GNU 7.4.0, closing=0, curl_closed=0, in_multi=1, paused=0, buffer_size_=1048576, buffer_offset_=655360, spill_.size()=16384, spill_offset_=0, total_downl    149 oaded_=35258286
150 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:135] (0x5566d178f090)  repeats=11313, crash_timeout=240m, (crash_deadline - now)=0s
151 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:160] (0x5566d178f090) effective_url=https://storage-download.googleapis.com/coryan-test-us-west1/ob-Pg7JNTdIeburqcPt.txt code=200, size_dow    151 nload=35258368, content_length=48207097
152 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:168] (0x5566d178f090) headers= accept-ranges=bytes cache-control=private, max-age=0 content-length=48207097 content-type=application/octet-    152 stream date=Thu, 18 Jul 2019 12:55:15 GMT etag="635ba29af76bb2c0ab2e81c0028022d2" expires=Thu, 18 Jul 2019 12:55:15 GMT http/1.1 200 ok
153 = last-modified=Wed, 26 Jun 2019 02:01:44 GMT server=UploadServer x-goog-expiration=Fri, 26 Jul 2019 02:01:44 GMT x-goog-generation=1561514504303332 x-goog-hash=crc32c=3DrUbw== x-goog-hash=md5=Y1uimvdrss    153 CrLoHAAoAi0g== x-goog-metageneration=1 x-goog-storage-class=REGIONAL x-goog-stored-content-encoding=identity x-goog-stored-content-length=48207097 x-guploader-uploadid=AEnB2UoWh4RxsnaT3IZgvzXPr5j_ajcWnhb    153 Y3n1t3GMClG6hHrhYvuJ39RohodQPIalvRyOBhVLO8yraL1oM1w3mrPewDHhqSg
154 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:175] (0x5566d178f090)  remote=<IP_ADDRESS>:443 [1bb]
155 [INFO /w/google/cloud/storage/internal/curl_download_request.cc:179] (0x5566d178f090)  local=<IP_ADDRESS>:52320 [cc60]
156   sl  local_address rem_address   st tx_queue rx_queue tr tm->when retrnsmt   uid  timeout inode
157    9: 0B19300A:CC60 808E7D4A:01BB 01 00000000:00000000 00:00000000 00000000     0        0 23550286 1<PHONE_NUMBER>000000 20 8 2 10 -1

I am attaching a better selection of the strace, note that the first time it prints "FATAL" the application is trying to poll:
 135270 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 135271 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 135272 nanosleep({tv_sec=0, tv_nsec=10000000}, 0x7fff167cc430) = 0
 135273 write(2, "[", 1)                        = 1
 135274 write(2, "FATAL", 5)                    = 5
 135275 write(2, "] ", 2)                       = 2
 135276 write(2, "Wait(), buffer_size_=1048576, bu"..., 211) = 211

And the previous call to read() blocked with EAGAIN:
 101332 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 1 ([{fd=7, revents=POLLIN|POLLRDNORM}])
 101333 read(7, "\27\3\3 \21", 5)               = 5
 101334 read(7, "1)d\275>fu\326\267\"\241\f\325\223q\363\240\3\223\373\265\342F\222\342\t\350U\vt\r_"..., 8209) = 7609
 101335 read(7, 0x5566d3915ee1, 600)            = -1 EAGAIN (Resource temporarily unavailable)
 101336 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)
 101337 poll([{fd=7, events=POLLIN|POLLPRI|POLLRDNORM|POLLRDBAND}], 1, 0) = 0 (Timeout)
 101338 poll([{fd=7, events=POLLIN}], 1, 10)    = 0 (Timeout)

To summarize: from the application's perspective, it called read() on a socket, that call returns EAGAIN, then the application polls that socket for at least 240s and no data shows up.  When the application gives up, /proc/self/net/tcp shows that the socket is empty, and it is in the ESTABLISHED state.
regression-1563454475-13337-5p6jz.strace.filtered.log.gz

I noticed that other sockets were using a lot of memory (some of them used 4MiB!).  I suspect the kernel is not accepting more data for the socket that is stalled because the TCP buffer memory is used by all the other sockets.
Reducing the maximum recv buffer size to 128 KiB made the problem go away.

Here is a log showing how much memory the other sockets are using for a single container, note that with 35 machines and 250 containers we should expect some machines to have 7 containers running.
regression-1563552296-8447-xm2sm.read-object-stall-regression.txt


[PREGEL] Tests for HITS and new implementation according to Kleinberg

Scope & Purpose
Adds the classical variant of the HITS algorithm (as HITSKleinberg by the name of the author)  and tests for it.

[ ] :hankey: Bugfix
[ ] :pizza: New feature
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification

Checklist

[ ] Tests

[ ] Regression tests
[ ] C++ Unit tests
[ ] integration tests
[ ] resilience tests

[ ] :book: CHANGELOG entry made
[ ] :books: documentation written (release notes, API changes, ...)
[ ] Backports

[ ] Backport for 3.9: (Please link PR)
[ ] Backport for 3.8: (Please link PR)
[ ] Backport for 3.7: (Please link PR)

Related Information

[ ] Docs PR:
[x] Enterprise PR: https://github.com/arangodb/enterprise/pull/1009
[ ] GitHub issue / Jira ticket:
[ ] Design document:

This needs a CHANGELOG entry as it adds a "new" Pregel algorithm

I added one.


mime type problem in Android R (11, API 30)

I got issue in Android R when I tried to set mime type as video and audio. using galleryMimeTypes().
the below code, works well in Android Q (10, API 29) and below, it is able to show video and audio files only. but it won't work in Android R (11, API 30), it shows photos only, not doesn't show video and audio files.
            ImagePicker.Companion.with(this)
                    .galleryOnly()
                    .galleryMimeTypes(new String[]{"video/*", "audio/*"})
                    .start();

the part of Manifest is below,
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

android:requestLegacyExternalStorage="true"

the part of gradle file is below, I didn't add florent37:inline-activity-result-kotlin
implementation 'com.github.dhaval2404:imagepicker:1.7.5'
compileSdkVersion 30
minSdkVersion 24
targetSdkVersion 30
buildToolsVersion '30.0.2'

Haha, I didn't know that the ImagePicker can also be used to Pick Audio & Video files. Just gave it a try on Android 11 and It seems to be working fine. I have tested in Pixel 2 Emulator(API 30). I will check this on other devices just to verify the issue.
Sorry for the late reply.

I have tested in Pixel 4a (with Android 11, API 30) and It's working fine. Please let me know the device to reproduce the issue. Looking forward to your reply. Thank you!

I have tested in Pixel 4a (with Android 11, API 30) and It's working fine. Please let me know the device to reproduce the issue. Looking forward to your reply. Thank you!

Thanks for the reply. I don't have a physical phone with Android 11. I use Google Play Intel x86 Atom_64 System Image to test. it doesn't pick audio and video file.

@jp-eagle Today, I will check on Google Play Intel x86 Atom_64 System Image.
Thank you so much for your reply.

any finding about this?

I believe this is the cause of the issue.

I managed to fix the compatibility issue by rolling back my SDK Target to 28, but this need to be fixed so it can be used with future SDKs in the future.

@jp-eagle Issue is resolved in ImagePicker v2.0. Please give it a try.

Hehehe.. I was finding answer to this question too.
Does this awesome library can work to pick any kind of file i.e audio, video, image :)


IndexError: index 320 is out of bounds for axis 0 with size 320

Hello,
I've been trying to train your KittiSeg network on some cell data.
I've taken the following steps to convert the network for my data:

changed the hypes image size
changed hypes background/foreground color
replaced the maybe_download_and_extract function

The network trains as it is supposed to, but when it runs the evaluation script, I get the following error:
/gpfs/gss1/home/frhaa13/DeepBacteriaSegmentation/network/incl/seg_utils/seg_utils.py:74: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 320 but correspondin$
  fnArray = cur_prob[(gtBin == True) & (validMap == True)]
/gpfs/gss1/home/frhaa13/DeepBacteriaSegmentation/network/incl/seg_utils/seg_utils.py:82: VisibleDeprecationWarning: boolean index did not match indexed array along dimension 0; dimension is 320 but correspondin$
  fpArray = cur_prob[(gtBin == False) & (validMap == True)]
W tensorflow/core/kernels/queue_base.cc:294] _0_Queues/fifo_queue: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
  File "train.py", line 133, in <module>
    tf.app.run()
  File "/gpfs/gss1/home/frhaa13/tensorenv/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 44, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "train.py", line 129, in main
    train.do_training(hypes)
  File "incl/tensorvision/train.py", line 396, in do_training
    run_training(hypes, modules, tv_graph, tv_sess)
  File "incl/tensorvision/train.py", line 289, in run_training
    hypes, sess, tv_graph['image_pl'], tv_graph['inf_out'])
  File "/gpfs/gss1/home/frhaa13/DeepBacteriaSegmentation/network/hypes/../evals/kitti_eval.py", line 127, in evaluate
    gt_image, output_im)
  File "/gpfs/gss1/home/frhaa13/DeepBacteriaSegmentation/network/hypes/../evals/kitti_eval.py", line 34, in eval_image
    validArea=valid_gt)
  File "/gpfs/gss1/home/frhaa13/DeepBacteriaSegmentation/network/incl/seg_utils/seg_utils.py", line 82, in evalExp
    fpArray = cur_prob[(gtBin == False) & (validMap == True)]
IndexError: index 320 is out of bounds for axis 0 with size 320

Nvm, some of my pictures were the wrong resolution


Package is using API from Combine with iOS 15 but Package.swift requires iOS 13

Minimum deployment version iOS should be iOS 15 because package is using Combine API with minimum iOS 15.
$_invocations.receive(on: queue).values
    public func verify(_ member: Member,
                       count: Count,
                       assertion: @escaping MockableAssertion,
                       timeout: TimeoutDuration,
                       file: StaticString = #file,
                       line: UInt = #line) async {
        do {
            let invocationsSequence = $_invocations.receive(on: queue).values
            try await withTimeout(after: timeout.duration) {
                for await invocations in invocationsSequence {
                    let matches = invocations.filter(member.match)
                    if count.satisfies(matches.count) {
                        break
                    } else {
                        continue
                    }
                }
            }
        } catch {
            let matches = invocations.filter(member.match)
            let message = """
            Expected \(count) invocation(s) of \(member.name) before \(timeout.duration)s, but was: \(matches.count)
            """
            assertion(count.satisfies(matches.count), message, file, line)
        }
    }

@Kolos65 I think if you want to support iOS 13.0 you will need to use a solution similar to what I did originally.

Already put up a fix in #42
The only problem was the values prop which could be easily implemented with an AsyncStream.

Fixed in 0.0.7


Can't disable first item on initiate

Not sure if intended, but I can't seem to disable the first option.
https://jsfiddle.net/uf9bmpy6/

I am having the same issues too:


ISPN-15317 Unable to query cached records with custom key objects tha…

https://issues.redhat.com/browse/ISPN-15317
The issue can be resolved by updating the SQL queries defined, no code changes are required.

Thanks, @ryanemerson!


Continue to do autocomplete after word is chosen

I have a case where I'm using ajax to provide autocomplete data. But if a keyword is chosen from the list, autocomplete stops until i press backspace. Space doesn't trigger autocomplete.
I'm changing the json based on the keyword, so when a keyword matches one in the json, I use that to generate a new json with the first keyword an add a new keyword behind with space as a divider.
first json used
{'name':'keyword11'}
{'name':'keyword12'}
{'name':'keyword13'}
{'name':'keyword14'}
second json used
{'name','keyword1 keyword21'}
{'name','keyword1 keyword22'}
{'name','keyword1 keyword23'}
so auto complete will show keyword1* first and when one is chosen, then it will show keyword1* keyword2*
if i just write the whole word the autocomplete works

Soved

can you explain how ??


⚠ ClamAV has degraded performance

In 3a4915f, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 84 ms

Resolved: ClamAV performance has improved in df69f69 after 6 minutes.


Android clear data function effects on Random IDs (closes #1067)

This PR implement text changes proposed in "FAQ: Effect of clear data on random ID storage" https://github.com/corona-warn-app/cwa-website/issues/1067 to FAQ articles:
(EN) Are the exposure logs/random IDs removed from my phone when I uninstall the app?
https://www.coronawarn.app/en/faq/#delete_random_ids
(DE) Werden die gesammelten Zufalls-IDs beim Deinstallieren der App entfernt?
https://www.coronawarn.app/de/faq/#delete_random_ids
Changes made

The FAQs are extended to describe that Random IDs are lost if the Android operating system storage "Clear data" function is applied to the "Corona-Warn" app. This could for example be through

Settings > Apps > "Corona-Warn"
Tap "Storage"
Tap "Clear data"
depending on Android version and implementation.

The paragraph referring to "reset app" is made clearer that it does not apply to "clear data" from the operating system.

English (EN)
Are the exposure logs/random IDs removed from my phone when I uninstall the app?
https://www.coronawarn.app/en/faq/#delete_random_ids

German (DE)
Werden die gesammelten Zufalls-IDs beim Deinstallieren der App entfernt?
https://www.coronawarn.app/de/faq/#delete_random_ids

References
corona-warn-app/cwa-app-android#642 (comment) from on Oct 30, 2020 @thomasaugsten wrote:

I doubled check and it looks Google is now registering the data deleting and turns off the exposure notification framework the leads to a key wiping. We will update our FAQ

Perhaps @thomasaugsten could review this PR for technical accuracy?

I changed "sie" to "Sie" and added a full-stop at the end of the sentence.

@MikeMcC399 @dsarkar
Just one point:
In the German text in the sixth paragraph, you refer to "Zufalls-IDs" whereas before it was always called "Begegnungsaufzeichnungen". "Zufalls-IDs" is introducing a new terminus in this section, what could lead to confusion. I'd suggest to keep it consistent. Personally, I like "Zufalls-IDs" more, as it better describes the data stored in ENS.
Would it make sense to add information at the end of the section, that installing another ENS-based contact tracing app (like Immuni and alike) before uninstalling CWA or deleting its data, prevents 'Random IDs" to be erased? So that this method could be used as a last resort in case of non-functional CWA without losing RPIs/TEKs?

@vaubaehn

In the German text in the sixth paragraph, you refer to "Zufalls-IDs" whereas before it was always called "Begegnungsaufzeichnungen". "Zufalls-IDs" is introducing a new terminus in this section, what could lead to confusion. I'd suggest to keep it consistent. Personally, I like "Zufalls-IDs" more, as it better describes the data stored in ENS.

This PR does not change the use of the terms "Begegnungsaufzeichnungen" and "Zufalls-IDs". The use is consistent with the definition in the app under Startseite > Dreipunkte-Symbol > Überblick

Would it make sense to add information at the end of the section, that installing (and activating) another ENS-based contact tracing app (like Immuni and alike) before uninstalling CWA or deleting its data, prevents 'Random IDs' to be erased? So that this method could be used as a last resort in case of non-functional CWA without losing RPIs/TEKs?

The Open Source Team decided in #320 to remove this information from the FAQ because it was not documented by Google.
So, in summary I don't plan further changes and I would be happy for this PR to be merged.

@MikeMcC399
I think I couldn't explain well enough what I mean. If it's about my bad expressions in English language, I beg your pardon in case I'm excessively straining your patience.
When we are talking about Rolling Proximity Identifiers (RPIs), in German language CWA app and CWA website refer to:

Begegnungsaufzeichnungen
Begegnungs-Aufzeichnung
Zufalls-IDs
rollierende Näherungs-IDs
if we refer to the link 'Datenbank und Schlüssel vom Gerät gelöscht' to Google's documentation within the section of this PR: Datenbank

... and probably more, which I couldn't spot within a quick research.
Moreover, Google names them inside the ENS UI: Zufallserkennungen.
What I was proposing to change, was to replace the 'Zufalls-IDs' in the sixth paragraph with 'Begegnungsaufzeichnungen' to have the same wording throughout the whole section. I felt irritated by the change of naming RPIs from 'Begegnungsaufzeichnungen' to 'Zufalls-IDs', and needed to think (again) if both words are meaning the same thing. I guess that a less technically advanced user might have even more problems to understand, what is talked about. Yes, you're right, there is a definition inside the CWA app, but I doubt that users, who want to understand if 'Zufalls-IDs' and 'Begegnungsaufzeichnungen' means the same, spontanously knew where to look it up (and do so).
So, changing the last 'Zufalls-IDs' to 'Begegnungsaufzeichnungen' (or all 'Begegnungsaufzeichnungen' in this section to 'Zufalls-IDs') would have made a difference for me with regard to readability, comprehensibility and style.
When I was reading the German section once again before writing this comment, I found that I overlooked the title of the German section before: 'Werden die gesammelten Zufalls-IDs beim Deinstallieren der App entfernt?' Following the title, the subject in this section handled 'Begegnungsaufzeichnungen' except inside the sixth paragraph. Together with the title, it's more clear, that 'Zufalls-IDs' also means 'Begegnungsaufzeichnungen'. However, I think the style can still be improved with minimum effort:
If we look to the English section, the title is 'Are the exposure logs/random IDs removed from my phone when I uninstall the app?'
Both synonyms are introduced within the title. Within the following text, both terms are used interchangingly. For me, in the English text it's much more clear, that both terms refer to the same object.
So, my (pendatic) final proposal to improve both English and German texts is:
Change English title to: 'Are the exposure logs (random IDs) removed from my phone when I uninstall the app?' - because the brackets give a better signal that 'exposure logs' and 'random IDs' are the same. The slash '/' often (but not always) refers to different objects that have the same importance.
Change the German title to: 'Werden die gesammelten Begenungsaufzeichnungen (Zufalls-IDs) beim Deinstallieren der App entfernt'? in line with the English title. I think, the following text is more clear then, and there is no need to replace any 'Begenungsaufzeichnungen' or 'Zufalls-IDs'.
I'm aware that you're not the creator of the original texts and the intentenion of this PR was a clarification in a different subject - but why not using the opportunity to improve style and comprehensibility?

Would it make sense to add information at the end of the section, that installing (and activating) another ENS-based contact tracing app (like Immuni and alike) before uninstalling CWA or deleting its data, prevents 'Random IDs' to be erased? So that this method could be used as a last resort in case of non-functional CWA without losing RPIs/TEKs?

The Open Source Team decided in #320 to remove this information from the FAQ because it was not documented by Google.

Yes, you are right. But if we relied on the (sketchy) documentation by Google always, things may be improved with an extraordinary delay, if ever. I'm trusting @kbobrowski 's findings here https://github.com/corona-warn-app/cwa-app-android/issues/1084#issuecomment-719642452 100%, and if I remember correctly, there were additional user reports in this context that confirmed this fact.  Hierfür können wir allerdings weder eine Bestätigung noch eine Gewährleistung geben.
 Eine Liste dafür geeigneter Kontaktverfolgungsapps anderer Länder finden Sie hier: https://developers.google.com/android/exposure-notifications/apps
English
According to individual reports from a few users, the exposure logs are not deleted when the data from the Corona-Warn-App is deleted via the Android system settings or when the Corona-Warn-App is uninstalled, if a second app for contact tracing from another country is on the device installed and activated, which is also based on the Exposure Notification Framework from Google. However, we can neither give a confirmation nor a guarantee for this.
Should you still have to delete the Corona-Warn-App data or have to uninstall the app, but you do not want to lose the random IDs, you can try this by installing a second app (at your own risk). A list of suitable contact tracing apps from other countries can be found here: https://developers.google.com/android/exposure-notifications/apps
However, I respect the fact, that this addition is not in the scope of your PR and could be seperated. On the other hand, your PR is a nice opportunity to add these findings. Feel free to use/adapt above text, in case you agree.

So, in summary I don't plan further changes and I would be happy for this PR to be merged.

When I'm submitting manuscripts to journals for review (or change requests on GH), I also don't plan any further changes to my work 😆 In best case there is a fast agreement of necessary (or possible) changes between submitter, referee and editor (on GH: product owner, here represented by @dsarkar ). So let's wait in modest patience for @dsarkar 's remarks.

@vaubaehn
I would defer to @heinezen on this topic since he is revising a lot of FAQ articles at this time and I am not the original author of the article.
I was attempting to make small changes to the article without questioning the base assumptions. I have marked up changes in the screen shots.
I think it would be preferable to sign off first on the small changes and take your points into consideration as a separate issue.

@MikeMcC399
Let's do like you suggested.

Hi @thomasaugsten , when you find time would be nice if you could approve @MikeMcC399 's PR for technical correctness. Please ignore my comments in this place, they are not associated with the technical content.
Thanks in advance!

@vaubaehn The translation of "RPI" used by us should always be "Zufalls-IDs", but has not yet been changed in all FAQ entries. However, we can consider changing the heading to make it more clear what is meant by that. I would do this in my PR for #971 .

Should you still have to delete the Corona-Warn-App data or have to uninstall the app, but you do not want to lose the random IDs, you can try this by installing a second app (at your own risk).

I'm not sure if we can just recommend a random app. Although it would work as a workaround of course.

@MikeMcC399 @vaubaehn @heinezen Thanks!


feat(webpack,cli): standalone build mode

Types of changes

[x] Bug fix (a non-breaking change which fixes an issue)
[x] New feature (a non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description

This PR implements @clarkdo idea (https://github.com/nuxt/nuxt.js/pull/4645#issuecomment-450365285) for standalone build mode and resolves #4645.
The build.standalone option is turned off by default to prevent any behaviour/breaking changes.
Benchmarks

Tested on bare-bone hello-world
Cold start time is measured using the production build of nuxt. From reading config until nuxt.ready()
Built on windows (Yeah I'm using windows again :P)

Normal build:

Size of server.js: 22 Kib
Cost of dependencies (vue, vue-router,  lodash (/omit), vue-no-ssr, debug): 7.4M
Cold start: 306.385ms
Memory usage: 29.9 MB (RSS: 106 MB)

Standalone build:

Size of server.js:  145 Kib
cold start: 306.550ms
Memory usage: 28.6 MB (RSS: 106 MB)

Results prove that there should be no big runtime diffs with the standalone mode. Still, we need to benchmark nuxt against real-world projects to decide on the default value.
Remarks

This is strange what dependency adds lodash/omit to the server bundle. Standalone build tree shakes only needed parts but should be still addressed
We need to announce users to use nuxt build --standalone when they need to use it with nuxt-start.

Alternative: For now we can add dependencies again to the nuxt-start distribution but it makes package bigger again

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly. (PR: #)
[ ] I have added tests to cover my changes (if not applicable, please state why)
[ ] All new and existing tests are passing.

Codecov Report

Merging #4661 into dev will decrease coverage by 0.03%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##              dev    #4661      +/-   ##
==========================================
- Coverage   90.82%   90.79%   -0.04%
==========================================
  Files          67       67
  Lines        2223     2226       +3
  Branches      543      545       +2
==========================================
+ Hits         2019     2021       +2
- Misses        185      186       +1
  Partials       19       19

Impacted Files
Coverage Δ

packages/config/src/config/build.js
85.71% <ø> (ø)
:arrow_up:

packages/webpack/src/config/server.js
100% <100%> (ø)
:arrow_up:

packages/cli/src/commands/build.js
80% <50%> (-3.34%)
:arrow_down:

packages/vue-renderer/src/renderer.js
95.1% <0%> (-0.55%)
:arrow_down:

packages/webpack/src/config/base.js
96.47% <0%> (+1.17%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d455408...8afb160. Read the comment docs.

In standalone mode, webpack will bundle all deps, but if 3rd party lib is using dynamic require, it may throw module not found error.
For example: consola requires winston, but winston is devDependency, so webpack will throw error when bundle winston into server bundle

@clarkdo Yes. That's a known problem of webpack with Node.js libraries like consola. We can whitelist some in standalone mode. But i don't recommend because it is dangerous in terms of a standalone build.

What about when someone dont want ( or just updated nuxt from 2.3 to 2.4 and it will break) to use standalone mode, but rather just use nuxt start as solo production dependency. Then there still will be problem from #4645.

@aldarund I just mentioned this in PR body as an alternative option:

Alternative: For now we can add dependencies again to the nuxt-start distribution but it makes package bigger again

But I assume it is a soft breaking change. We can make nuxt-start smarter to detect non-standalone builds and warn users.

Updates:

Dependencies for alternative approach added right into nuxt-start package (c664e3df7b2175d23e88e8c2df95c85ce005fcb1, https://github.com/nuxt/nuxt.js/commit/b5a0f788dc01ce60403835ffb59fa73978c21179)

The mentioned extra import for lodash removed from server.js. (https://github.com/nuxt/nuxt.js/commit/6178c470ba81367e0c9f856b38781d270d65edfe). Reducing hello world's server size to 124 KiB (Saving 21Kib)

@pi0, could you describe the current status of this future? It looks undocumented now.

Hi @Norserium it is indeed stable and being used for vercel serverless builder for a long time. I think there should be an open issue for documenting /cc @debs-obrien

@pi0, yes it's. But it's defined as an experimental mode.

That's why issue reopened ;)

@pi0 how should I work with standalone option when modules defined in nuxt.config.js are required on demand ? Do we have some "whitelist" do tell the builder we want to use these specific module in standalone mode ?

@pi0 how should I work with standalone option when modules defined in nuxt.config.js are required on demand ? Do we have some "whitelist" do tell the builder we want to use these specific module in standalone mode ?

@mercs600 You can instead move build-only modules to buildModules and list runtime modules in modules :)

@mercs600 You can instead move build-only modules to buildModules and list runtime modules in modules :)

right! I forgot about buildModules, thanks ✋

right! I forgot about buildModules, thanks ✋


proposal: encoding/json: Make encoders set Content-Type.

What we do
Currently encoding a JSON to a http.ResponseWriter sets a plaintext content-type:
func handler(w http.ResponseWriter, r *http.Request) {
	e := json.NewEncoder(w)
	err := e.Encode(map[string]string{"foo": "bar"})
	if err != nil {
		panic(err)
	}
}

func main() {
	r, w := httptest.NewRequest("GET", "/", nil), httptest.NewRecorder()
	handler(w, r)
	fmt.Println(w.Header().Get("Content-Type")) // text/plain; charset=utf-8
}

playground
This happens on a response recorder the same way it happens on a standard http serving stack.
The issue is that when content-type is not explicitly set, http uses http.DetectContentType on the first slice passed to Write, which will sniff as plaintext since there is no good way to sniff JSON.
What we could do
I propose to make (*json.Encoder).Encode() calls check if the passed argument has a Header() http.Header method, and if so set the appropriate content-type.
Cons:

encoding/json will depend on net/http, which seems less than desirable
It will make encoding slightly slower as it adds a type inference call to Encode calls

Pros:

Setting the appropriate content-type is more correct (while technically JSON is also plain text, "application/json" would be more appropriate)
Browsers implement some security features to protect sensitive data from being leaked cross-site, and those features heavily rely on content-type (e.g. CORB)

/cc @mvdan @rsc @dsnet @bradfitz as per owners.

there is no good way to sniff JSON.

Could you expand a bit on this, for the sake of clarity?

encoding/json will depend on net/http, which seems less than desirable

I share this concern; teaching encoding/json about net/http sounds wrong :)
Also, have you thought about fixing this via static analysis? It would probably not meet the inclusion bar for vet, since I doubt this problem is common and important enough, but it could perhaps be added to staticcheck. With its fairly advanced code analysis, it should be possible to detect http handler funcs which write JSON but never set a content-type, even if there are func calls in between. CC @dominikh in case he's interested in new security checks.

Related: #10630.

Could you expand a bit on this, for the sake of clarity?

Sniffing is cropped at 512 bytes for performance reasons, and you cannot know if a JSON is a JSON until you have tokenized the entire thing.

I share this concern; teaching encoding/json about net/http sounds wrong :)

Yup, this is my biggest concern TBH.

Also, have you thought about fixing this via static analysis?

It seems a mess to check if one of the handlers in a chain of handlers and decorators set the CT at one point. The whole decoration chain could be set up at runtime so I think it might even be impossible.

What we could do
I propose to make (*json.Encoder).Encode() calls check if the passed argument has a Header() http.Header method, and if so set the appropriate content-type.

Instead of using the Encoder we could have the json.NewEncoder check:
if _, ok := w.(http.ResponseWriter); ok {
   // Set something on the encoder so it knows what to do.
}

This would avoid possibly the cons number 2. but number 1. would still hold true and json would know about http, which I agree on your point 1.

Cons:

encoding/json will depend on net/http, which seems less than desirable
It will make encoding slightly slower as it adds a type inference call to Encode calls

It would feel somewhat more natural to have a http.NewJSONEncoder that wraps json.NewEncoder and knows what to do with http headers when dealing with JSON, but I guess there you have no guarantee that people would use it: is this why this option might now be a good one?

JSON comes from many places; tying this to the encoder seems like a mistake.
Also lots of non-JSON is valid JSON, such as the one-word English text true.
Can you say more about the Pros? I am not too worried about "technically more correct".
What are the security features that are defeated by sending JSON as text/plain?

The more I think about this the more cons I see.
The security feature I was referring to is Cross-Origin-Read-Blocking, but even if the specification doesn't say so, browsers still seem to protect text/plain if it sniffs as JSON (I had to do some experimentation).
Since I am also not really interested in the correctness of the CT and the security benefit seems to be none, I am closing this one.
Thanks everyone for the inputs.


cuda-mxnet build failed for the past 2 months

From this page, it seems that the dockerfile fails to compile recent mxnet updates:
https://hub.docker.com/r/kaixhin/cuda-mxnet/builds/

Not sure exactly what is causing the error, but it builds fine on my machine, and even on the Docker Hub (apparently). Seems like something is going wrong when pushing the image, but we will have to wait for Docker to respond: https://github.com/docker/hub-feedback/issues/547


hide_gitignored takes precedence over hide_by_* and never_show*

It seems that the hide_gitignored = false option takes precedence over hide_by_name, hide_by_pattern, never_show and never_show_by_pattern, which I believe to be counter-intuitive.
My situation is that I have a javascript project with a .gitignored node_modules folder. I want to be able to hide the node_modules folder while still keeping other .gitignored files visible.
I also have .DS_Store set to never_show, but this again doesn't work if .DS_Store is .gitignored.
As soon as I remove node_modules or .DS_Store from the .gitignore file they then get hidden.
The issue appears with hide_by_pattern and never_show_by_pattern as well.
These are the relevant config lines:
{
  filesystem = {
    filtered_items = {
      hide_dotfiles = false,
      hide_gitignored = false,
      hide_by_name = {
        ".git",
        "node_modules",
      },
      never_show = {
        ".DS_Store",
      },
    },
  }
}

I would expect all files in hide_by_name, hide_by_pattern, never_show and never_show_by_pattern to be hidden irrespective of the hide_gitignored option. This is what happens with hide_dotfiles - hiding the .git directory works even though I have hide_dotfiles = false.
This is a similar issue to #682.

Reproducible, and I think so too.

It seems that the hide_gitignored = false option takes precedence over hide_by_name, hide_by_pattern, never_show and never_show_by_pattern, which I believe to be counter-intuitive.

Do you @cseickel agree on this? If so, I'll work on it.

Closing with #946


Feature request: format compile errors by files in src

When I get a compile error, I typically want to know where the error is in files in src, not the cascade into the libraries. Novices sometimes don't even know there is useful information in the 'ng serve/build' terminal. I think errors could be understood a lot easier if the errors were formatted something like this:
ng build ->
Typical Error Message:
...business as usual...
Compile Errors from src files:
[file_name] [line number] [error message]
I think this would help everyone search out the most useful parts of the message, and help novices discover important information.

https://github.com/angular/angular-cli/pull/1455 made these better, since we use the webpack report format.


PubSub

Design doc is here.

[x] #1594
[x] #1599
[ ] https://github.com/TBD54566975/ftl/pull/1608
[ ] #1607
[ ] #1595
[ ] #1596
[ ] #1598

Todos:

make sure other declarations (like verbs) make an error if //ftl:export is used on them, guiding them to correct usage...

Case to handle:

Deployment registers a topic with event A
Updated deployment happens, now with the topic defined with event B
... what should happen?

Case to handle:

Deployment registers a subscription

Some events are consumed by the subscription. Curser points to event row A

Replace deployment, now without the subscription

Quick thoughts: Does that subscription row just sit there untouched? Should we be deleting it? Will this work ok when there are multiple versions of a module? Don't worry about that for now?

... Wait a week
Replace deployment again, now with a subscription with the same name as the old subscription
... is the cursor still pointing to a week ago?

Thing to think about:
When a module is updated and removes a topic, should the topic just sit there?

The module won't be able to publish any new events anyway...
It allows time for subscribers to catch up to the remaining unhandled events, which is good

Case to handle:

Can not publish to external module's topic

Queries are doing a lot of joins. Should we denormalize module name into topics?

For now we won't worry about catching any of these cases, we want a holistic approach to managing schema changes across FTL.

Queries are doing a lot of joins. Should we denormalize module name into topics?

Yeah, we do it in other places, so if you think it's necessary go for it.

I'm going to close this out. We can revisit later if we decide to do batching. Nice work Matt.


⚠ Bofrost.lu has degraded performance

In 51203a7, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1557 ms

Resolved: Bofrost.lu performance has improved in 9fe9794 after 43 minutes.


Added re-crawl immediately and refactored JS code

Optimized JS code, fixed tool to work with 7.2 onwards versions, added re-crawl immediately button, added link to most recent documentation.

What testing have you done for this change?

To test the changes, I pushed sample URL using "Delete", "Re-Crawl" and "Re-crawl now" buttons. Verified the feeds were generated correctly, accepted by the GSA and respective action was performed by the GSA.


CurrentCulture=en-US if "console": "integratedTerminal" in debugger

Does this issue occur when all extensions are disabled?: Yes/No
I couldn't run debugger if all extensions are disabled.

VS Code Version: 1.85.1
OS Version: MacOS BigSur 11.7.10

Steps to Reproduce:
I added a line to print out the CurrentCulture in Program.cs in the attached test project (created with "dotnet new webapi").
When debugging the project (Press F5):
If I commented out the line "console": "integratedTerminal" in the launch.json, the CurrentCulture=en-GB.
If I uncommented out the line, the CurrentCulture=en-US.
If I run "dotnet run" in the macOS terminal the CurrentCulture is always en-GB regardless of whether "console": "integratedTerminal" is added or not.
en-GB is what I expected as the Region in the system pref of my laptop is configured as UK.
testwebapi.zip

I'm guessing this is related to your terminal/shell confguration in vscode, not vscode's terminal, but I don't know how this is determined on mac

This type of issue is typically related to environment differences between VS Code's terminal and the external terminal.
Closing this off as it doesn't look actionable on our end, you can read more about our issue process here. Thanks for understanding.
/label *question


Can you use a Managed Service Account for the gateway?

Running through the install - I have a gMSA all setup and ready to go - but the app always requires a password. This doc makes it seem like a managed service account can be used (preferred, even).
It might need to be made clearer if this is the case.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 285b3c81-1498-3497-9021-84227e7f280b
Version Independent ID: 20064c8f-87fa-ed89-a788-8c9b34968d4d
Content: Change the on-premises data gateway service account
Content Source: data-integration/gateway/service-gateway-service-account.md
Technology: on-premises-data-gateway
GitHub Login: @arthiriyer
Microsoft Alias: arthii

Likewise. Server team have kindly supplied me with a GMSA for running this service, but looks to me like they aren't compatible.

We have just recently introduced support for service principal via the silent install https://powerbi.microsoft.com/en-us/blog/announcing-automation-of-data-gateway-installationpublic-preview/

@jameslloyd3 Closed after being answered.


Add integration tests for ODM features

Same as #39 but without extra composer deps
Had to leave 'config.mongodb' in composer json so that it would allow installing doctrine-odm

The travis job has failed because of some travis issue, but you can see in the php 7.1 job that it passes.

Please rebase on top of master, it conflicts now.

Thanks!


adding close to Asciidoctor to avoid to leak when creating an asciidoctor instance

Currently Asciidoctor instance can't be released.
With JRuby it means at least 2 pools are leaking each time an instance is created.
It leads to a lot of issues in several environments.
This PR adds a close() method to Asciidoctor and ensures JRuby instance is properly released when called.
Saved ~1mn on 1mn16s of execution with exec-maven-plugin for instance and avoids leakage in EE context.

FWIW, a JRuby instance that becomes GCable will eventually shut itself down, but we strongly recommend tearing it down once the instance is no longer needed.
PR seems 👍 to me.

Thanks!
Just have to remember that the next release has to be 2.2.0 instead of 2.1.1.


Prepare for HL7 FHIR Connectathon

Problem statement
ReportStream engineers will be attending the FHIR Connectathon. The track includes intake of an at-home COVID test in FHIR format then delivery to a downstream system. The intake and delivery of a sample message is useful to determine current readiness of ReportStream for this effort.
What you need to know
Example FHIR message. You can find more here.
We will use APIKey auth instead of the smart-on-fhir server-server auth since that is easier for folks to deal with.
UP currently only supports sending in HL7 (See FHIRTranslator.doWork). Inside that function, instead of converting bundle to HL7, we want to do something like this, probably: FhirContext.forR4().newJsonParser().encodeResourceToString(updatedBundle)
PLAN:

(REQUIRED) Complete https://github.com/CDCgov/prime-reportstream/issues/9144 to allow sending FHIR. This is currently assigned to Jimmy from the platform team as part of ETOR work.
(NICE TO HAVE) Create special branch once 9144 is complete to insert Provenance resource if message does not have it. RS will throw an error during route step if Provenance does not exist since it uses it for routing. If we don't do this, we will have to manually edit messages before sending them to RS.
Create Demo environment for the special branch to be used by connect-a-thon. Questions: Will this work? Firewalls?
Test sample message. Is it unchanged as it moves through the pipeline?
Look at the test scenarios. Have a sample postman call with ApiKey senders can share.

At-Home In-Vitro Info Site
Acceptance criteria

[ ] ...

To do

[ ] ...

@brick-green can we close this?


Build Error: "AVAudioSession is unavailable on macOS"

OS: macOS 11b3
Xcode version 12 beta 3 (12A8169g)
Seems to be tied to AVKit the VideoPlayer SwiftUI view.

Detailed error messages:
/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.0.sdk/System/Library/Frameworks/_AVKit_SwiftUI.framework/Modules/_AVKit_SwiftUI.swiftmodule/x86_64-apple-macos.swiftinterface:4:8: Failed to build module 'AVFoundation' from its module interface; the compiler that produced it, 'Apple Swift version 5.3 (swiftlang-1<IP_ADDRESS> clang-1<IP_ADDRESS>)', may have used features that aren't supported by this compiler, 'Apple Swift version 5.3 (swiftlang-1<IP_ADDRESS> clang-1<IP_ADDRESS>)'
The above seems like it is a minor Swift compiler mismatch, but interested to see if others have seen this. If this is an Swift bug I will file Feedback.
Steps to reproduce:

Clone repository
Update Development team
Attempt to Build

In beta 3 you'll have to comment some interface in .swiftinterface AVFoundation system framework.
https://twitter.com/dimillian/status/1286265975969587200?s=21

That did the trick, thanks!


Multiple/Tagging - If a user hits the spacebar, adds characters, and hits enter, a blank tag can created.

This is obviously quite the edge case. A user can however, when entering a good deal of tags, find themselves in this situation, where a blank tag is created, then due to it being blank, the user isn't allowed to delete it. This could be solved by a variety of fixes, not sure what would make the most sense. Limit the speed, limit spaces, or to allow the developer to give a regexp/pattern.
Attached is an example, I've confirmed the issue with the newest version as well.

Can anyone give me any guidance or update to when this may be resolved?

Can you post a Plunker reproducing this?

Plunker with the bug.
http://plnkr.co/edit/z8KDrqwFUGbO2jWRA5Ni?p=preview

Oh dang, great work on the plunker @lkuklis. That made things much easier to reproduce on my end at least. What would it take to get a PR to fix this bad boy edge case? I'd be interested to help if needed. Thanks again! 👍 🙏


Add header for projectedfslib

This header allows providers like VFSForGit to create virtual
filesystems in userspace.
Copied from SDK version 10.0.17763.0.

This may address #759

Hi, check out my pull request at #910 :)


can you please tell me how can i plot the predicts like your paper image?

thanks for your repo very much, its very nice to read.

Hi, please refer to https://github.com/j96w/DenseFusion/issues/52.


Player position changed when bumping into borders of tilemap

Hello,
thank you for such a wonderfully easy plugin to use. It does everything I need and more
the only problem I have found is that when my player walks to the edge of the tilemap and collides with the border. The position of the player is updated and thus unaligns with the tilemap.
I'm sure there is a solution, but I am not familiar enough with phaser or your plugin to figure it out, my apologies.

In the top picture the player can be seen between two tiles

Hey @tylerroost,
thank you for the positive feedback!
I am glad that you find it useful and it is always nice to get feedback from people who use it in their projects. You are a great help to make this plugin better.
Let me take a look at the issue.

Hey @tylerroost,
thank you for the positive feedback!
I am glad that you find it useful and it is always nice to get feedback from people who use it in their projects. You are a great help to make this plugin better.
Let me take a look at the issue.

@tylerroost
I can't reproduce the issue. Can you provide more information to help me tracking it down?
i.e:

Version of the plugin
code that you used to initialize the plugin, including initialization of the sprites.

Thank you 🙂

@tylerroost
I can't reproduce the issue. Can you provide more information to help me tracking it down?
i.e:

Version of the plugin
code that you used to initialize the plugin, including initialization of the sprites.

Thank you 🙂

It may be from the fact that my tileset is 32x32, where as my player sprite is 64x64, thus there maybe half(32) tiles that allow the character to reposition.

It may be from the fact that my tileset is 32x32, where as my player sprite is 64x64, thus there maybe half(32) tiles that allow the character to reposition.

If this seems like a reasonable explanation let me know, and if you still want code I would be happy to provide

If this seems like a reasonable explanation let me know, and if you still want code I would be happy to provide

Plugin version: 1.2.2
var config = {
    type: Phaser.CANVAS,
    plugins: {
      scene: [
        {
            key: "gridMovementPlugin",
            plugin: GridMovementPlugin,
            mapping: "gridMovementPlugin",
        },
        {
            key: 'rexUI',
            plugin: rexuiplugin,
            mapping: 'rexUI'
        }
      ],
    },
    scale: {
      width: 720,
      height: 528,
      autoCenter: Phaser.Scale.CENTER_BOTH,
    },
    parent: 'content',
    pixelArt: true,
    physics: {
        default: 'arcade',
        arcade: {
            gravity: { y: 0 }
        }
    },
    scene: [
        BootScene,
        WorldScene,
        BattleScene,
        BattleUIScene
    ],
    scale: {
        mode: Phaser.Scale.FIT,
        autoCenter: Phaser.Scale.CENTER_BOTH,
    },
};

Here you can see the config for my Phaser.Game object, where the plugin is being invoked.
class WorldScene extends Phaser.Scene{
    constructor()
    {
        super('WorldScene')
    }

    WorldScene()
    {
        Phaser.Scene.call(this, { key: 'WorldScene' });
    }
    preload()
    {
        this.load.tilemapTiledJSON('WorldMap', 'assets/locations/WorldMap5.json');
        this.load.spritesheet('HGSSTileset',
            'assets/locations/HGSSTileset.png',
            { frameWidth: 32, frameHeight: 32 }
        );
        this.load.spritesheet('boy',
            'assets/Overworld/PlayerSprites/boy_run.png',
            { frameWidth: 64, frameHeight: 64 }
        );
        this.load.spritesheet('girl',
            'assets/Overworld/PlayerSprites/girl_run.png',
            { frameWidth: 64, frameHeight: 64 }
        );
        this.load.spritesheet('gen1SpriteMap',
            'assets/Pokemon/SpriteMaps/gen1SpriteMap.png'),
            { frameWidth: 128, frameHeight: 128 }
    }
    create()
    {
        var map = this.make.tilemap({ key: 'WorldMap' });
        var tiles = map.addTilesetImage('HGSSTileset');

        var ground = map.createLayer('GroundLayer', tiles, 0, 0);
        var grass = map.createLayer('GrassLayer', tiles, 0, 0);
        var buildings = map.createLayer('BuildingsLayer', tiles, 0, 0);
        ground.scale = 1;
        ground.setDepth(0)
        grass.scale = 1;
        grass.setDepth(1)
        buildings.scale = 1;
        grass.setDepth(2)
        buildings.setCollisionByExclusion();
        this.player = this.physics.add.sprite(400, 350, 'girl', 1);
        this.player.scale = 1;
        this.physics.world.bounds.width = map.widthInPixels;
        this.physics.world.bounds.height = map.heightInPixels;
        this.player.setCollideWorldBounds(true);
        this.cameras.main.startFollow(this.player);
        const gridMovementConfig = {
            characters: [
                {
                    id: "player",
                    sprite: this.player,
                    characterIndex: 0,
                    speed: 4,
                    startPosition: new Phaser.Math.Vector2(17, 17),
                },
            ],
        firstLayerAboveChar: 3,
        };

        this.gridMovementPlugin.create(map, gridMovementConfig);
    }
    update(time, delta)
    {
        const cursors = this.input.keyboard.createCursorKeys();
        if (cursors.left.isDown) {
            this.gridMovementPlugin.moveLeft("player");
        } else if (cursors.right.isDown) {
            this.gridMovementPlugin.moveRight("player");
        } else if (cursors.up.isDown) {
            this.gridMovementPlugin.moveUp("player");
        } else if (cursors.down.isDown) {
            this.gridMovementPlugin.moveDown("player");
        }
    }
};

Here is the scene that was used in the imagery. I have attached all necessary files
GridMovementBug.zip
If this is insufficient to duplicate please let me know what else you need.

Plugin version: 1.2.2
var config = {
    type: Phaser.CANVAS,
    plugins: {
      scene: [
        {
            key: "gridMovementPlugin",
            plugin: GridMovementPlugin,
            mapping: "gridMovementPlugin",
        },
        {
            key: 'rexUI',
            plugin: rexuiplugin,
            mapping: 'rexUI'
        }
      ],
    },
    scale: {
      width: 720,
      height: 528,
      autoCenter: Phaser.Scale.CENTER_BOTH,
    },
    parent: 'content',
    pixelArt: true,
    physics: {
        default: 'arcade',
        arcade: {
            gravity: { y: 0 }
        }
    },
    scene: [
        BootScene,
        WorldScene,
        BattleScene,
        BattleUIScene
    ],
    scale: {
        mode: Phaser.Scale.FIT,
        autoCenter: Phaser.Scale.CENTER_BOTH,
    },
};

Here you can see the config for my Phaser.Game object, where the plugin is being invoked.
class WorldScene extends Phaser.Scene{
    constructor()
    {
        super('WorldScene')
    }

    WorldScene()
    {
        Phaser.Scene.call(this, { key: 'WorldScene' });
    }
    preload()
    {
        this.load.tilemapTiledJSON('WorldMap', 'assets/locations/WorldMap5.json');
        this.load.spritesheet('HGSSTileset',
            'assets/locations/HGSSTileset.png',
            { frameWidth: 32, frameHeight: 32 }
        );
        this.load.spritesheet('boy',
            'assets/Overworld/PlayerSprites/boy_run.png',
            { frameWidth: 64, frameHeight: 64 }
        );
        this.load.spritesheet('girl',
            'assets/Overworld/PlayerSprites/girl_run.png',
            { frameWidth: 64, frameHeight: 64 }
        );
        this.load.spritesheet('gen1SpriteMap',
            'assets/Pokemon/SpriteMaps/gen1SpriteMap.png'),
            { frameWidth: 128, frameHeight: 128 }
    }
    create()
    {
        var map = this.make.tilemap({ key: 'WorldMap' });
        var tiles = map.addTilesetImage('HGSSTileset');

        var ground = map.createLayer('GroundLayer', tiles, 0, 0);
        var grass = map.createLayer('GrassLayer', tiles, 0, 0);
        var buildings = map.createLayer('BuildingsLayer', tiles, 0, 0);
        ground.scale = 1;
        ground.setDepth(0)
        grass.scale = 1;
        grass.setDepth(1)
        buildings.scale = 1;
        grass.setDepth(2)
        buildings.setCollisionByExclusion();
        this.player = this.physics.add.sprite(400, 350, 'girl', 1);
        this.player.scale = 1;
        this.physics.world.bounds.width = map.widthInPixels;
        this.physics.world.bounds.height = map.heightInPixels;
        this.player.setCollideWorldBounds(true);
        this.cameras.main.startFollow(this.player);
        const gridMovementConfig = {
            characters: [
                {
                    id: "player",
                    sprite: this.player,
                    characterIndex: 0,
                    speed: 4,
                    startPosition: new Phaser.Math.Vector2(17, 17),
                },
            ],
        firstLayerAboveChar: 3,
        };

        this.gridMovementPlugin.create(map, gridMovementConfig);
    }
    update(time, delta)
    {
        const cursors = this.input.keyboard.createCursorKeys();
        if (cursors.left.isDown) {
            this.gridMovementPlugin.moveLeft("player");
        } else if (cursors.right.isDown) {
            this.gridMovementPlugin.moveRight("player");
        } else if (cursors.up.isDown) {
            this.gridMovementPlugin.moveUp("player");
        } else if (cursors.down.isDown) {
            this.gridMovementPlugin.moveDown("player");
        }
    }
};

Here is the scene that was used in the imagery. I have attached all necessary files
GridMovementBug.zip
If this is insufficient to duplicate please let me know what else you need.

I very much appreciate it, though if I were to guess it has to do with the tilemap being nondivisible by the size of the player character. If this is in fact the cause I'm guessing a simple check against the divisible boundaries of the tilemap could lead to a solution.

I very much appreciate it, though if I were to guess it has to do with the tilemap being nondivisible by the size of the player character. If this is in fact the cause I'm guessing a simple check against the divisible boundaries of the tilemap could lead to a solution.

I assumed the same first. But it had nothing to do with it. Only the tile size of the map is important. However, I think I found the issue:
The problem is this part of your code: this.player.setCollideWorldBounds(true);
It enables the collision of the arcade physic engine. So this means the sprite will collide with the boundaries of your map on a per-pixel level. I recommend to just delete this line, because what you might want to have in a tile-based movement are tile-based collisions.
phaser-grid-movement-plugin does support collisions on a tile level. Your tiles just need a custom boolean property called collides with the value true. In case you are using Tiled you can simply add that property to your tileset. I described it in my article here: https://medium.com/swlh/grid-based-movement-in-a-top-down-2d-rpg-with-phaser-3-e3a3486eb2fd
Moreover, boundaries of the whole map are detected by the plugin by default (they are not tiles so they don't need a collides property.
Please let me know if that solved your problem 🙂

I assumed the same first. But it had nothing to do with it. Only the tile size of the map is important. However, I think I found the issue:
The problem is this part of your code: this.player.setCollideWorldBounds(true);
It enables the collision of the arcade physic engine. So this means the sprite will collide with the boundaries of your map on a per-pixel level. I recommend to just delete this line, because what you might want to have in a tile-based movement are tile-based collisions.
phaser-grid-movement-plugin does support collisions on a tile level. Your tiles just need a custom boolean property called collides with the value true. In case you are using Tiled you can simply add that property to your tileset. I described it in my article here: https://medium.com/swlh/grid-based-movement-in-a-top-down-2d-rpg-with-phaser-3-e3a3486eb2fd
Moreover, boundaries of the whole map are detected by the plugin by default (they are not tiles so they don't need a collides property.
Please let me know if that solved your problem 🙂

O god how could I have missed that. I'll look into it, and get back to you.

O god how could I have missed that. I'll look into it, and get back to you.

@tylerroost can you already give me any feedback whether the issue is resolved?

My apologies, Ive been swamped with work for school and have an interview on Friday, I should have some free time this weekend. I will schedule in time on Saturday. But based on your analysis, I would say your conclusion is fairly accurate. Like you said the plugin detects collision with the world boundary by default.
I just did some further research and I don't know if this is relevant, but it appears that when adding a body to a container it may reset one of these two properties
this.player.body.setCollideWorldBounds(true);
this.player.body.onWorldBounds = true;

Though it seems like I am colliding with the world boundary, so I'm not entirely sure what is going on, my best guess, without looking at the code for the plugin is that because I never set onWorldBounds to true, the plugin is not being notified that I am at the worldBounds. The other possibility is that even if I did set this property, if I added the player to a container it is implicitly resetting the property, which means the plugin is not being notified of the collision.
Though like you said the plugin is programmed in such a way that collision with the world bounds is enabled by default, and setting setCollideWorldBounds to true is detecting pixel level collisions and not tile level collisions like I want. I would argue the intended result of setting setCollideWorldBounds to true, while using your plugin is such that the plugin should override the behavior and thus be able to handle the collision. I'm not sure how you would go about this, but let me lay out an example to illustrate my point. Say I have many body gameObects in a container, without a deeper understanding of the plugin, I and possibly others would expect that setting setCollideWorldBounds to true would produce the intended tile based behavior.
TL;DR
There could be multiple reasons for why I am colliding at a pixel level with the boundary.
My intended behavior is more or less the typical expectation and thus should be factored into design for the plugin. In my mind this is easily accomplishable by setting the property (setCollideWorldBounds) to false. Though I am sure there are arguments against this. My apologies, if this comes off as presumptuous.

No problem. Don't rush.
Thank you for the detailed answer. I agree with you that it may be misleading for many users that the setCollideWorldBounds property has not the desired effect when using the plugin.
I think I will try to turn it off in the plugin by default (or find a different way of ignoring it). If some users need to turn it back on (for whatever reason) I can still add a configuration setting for this purpose.
I keep this issue up to date.

Sounds good, thank you for considering my request.

I looked a bit into it and thought more about it. I am not sure anymore if this is the correct way to go for the following reasons:
The property only exists on arcade physics objects. I think in most cases mixing up arcade physics with grid physics is a bad idea. Even if I check for this property and disable it, all the other behavior that comes with arcade physics would still be in place (for instance the collision with other arcade physics objects).
The plugin works when just using a plain sprite (Phaser.GameObjects.Sprite). Because the arcade sprite (Phaser.Physics.Arcade.Sprite) extends Phaser.GameObjects.Sprite, you can use it with the plugin. However, that does not mean you should. There might be good reasons to use an arcade sprite (maybe you already have one in your project?).
But when you mix in another physics engine then the borderCollision is not the only thing I would have to deal with.
Therefore I better don't mess around with the physics settings in the plugin, because if someone uses an arcade sprite there might be a specific reason for this.
The problem that you have with turning the setting back on is a separate issue I think.  To avoid that I would have to repeatedly turn it off, wich feels hacky and might also be confusing for a user who wants to enable it on purpose (for whatever reason).
In your specific case, would it be an option to use a plain Phaser.GameObjects.Sprite for your player or do you need the arcade physics?

Hmm, it does seem quite hacky as you described it, I need to do more research to determine the nature of my sprite. I will add it to my list of things to do this weekend.

I hope it's ok if I close this issue for now? I think it is not a real issue with the plugin but rather a clash of settings when using arcade physics with it. If you find out how we could make it more compatible feel free to reopen this issue or create a new one with the suggestion.


[BUG] Error communicating with API: 401 Client Error: Unauthorized for url: https://api.todoist.com/rest/v2/tasks

Installed from HACS.
Added to configuration.yaml and secrets.yaml - double and triple checked values.
I get this error.
I tried making a manual GET req to my API to verify my key/token, and it worked (I got a 200 response, with the tasks on my list)

configuration.yaml
# Loads default set of integrations. Do not remove.
default_config:

# Load frontend themes from the themes folder
frontend:
  themes: !include_dir_merge_named themes

automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

sensor:
  - platform: rest
    name: To-do List
    method: GET
    resource: 'https://api.todoist.com/sync/v9/projects/get_data'
    params:
      project_id:<PHONE_NUMBER>
    headers:
      Authorization: !secret todoist_api_token
    value_template: '{{ value_json[''project''][''id''] }}'
    json_attributes:
      - project
      - items
    scan_interval: 30

rest_command:
  todoist:
    method: post
    url: 'https://api.todoist.com/sync/v9/{{ url }}'
    payload: '{{ payload }}'
    headers:
      Authorization: !secret todoist_api_token
    content_type: 'application/x-www-form-urlencoded'

secrets.yaml
# Use this file to store secrets like usernames and passwords.
# Learn more at https://www.home-assistant.io/docs/configuration/secrets/
some_password: welcome

todoist_api_token: 'Bearer MY-KEY-TESTED-AND-VERIFIED'

The error shown in your screenshot is not related to this card, as it doesn't use that Todoist API endpoint.

Oh, sorry I got things mixed up - working fine now!


Network Failure Unhandled

Please describe the problem you are having in as much detail as possible:
An unhandled exception, as follows, is thrown when my network adaptor is disabled. This seems odd, as I was under the impression that this should be handled gracefully by the disconnect event.
Error: getaddrinfo ENOENT gateway.discord.gg:443
    at Object.exports._errnoException (util.js:1034:11)
    at errnoException (dns.js:33:15)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:73:26)

Include a reproducible code sample here, if possible:
The exception is thrown whenever there is any active connection to the Discord API, such as below.
const Discord = require('discord.js');
let robot = new Discord.Client();
let token = 'example123';
robot.login(token);

Further details:

discord.js version: 11.1.0.

node.js version: 7.8.0.

Operating system: Windows 10 Pro 10.0.15063 Build 15063.

Priority this issue should have – please be realistic and elaborate if possible: Medium. This exception may not be thrown very often, but if the intention is for disconnect to handle it, it could cause ungraceful failures in some cases, which would seem undesirable.

[x] I found this issue while running code on a bot account

[ ] I have also tested the issue on latest master, commit hash:

this is your internet, not a lib problem

@devsnek Well, in this case, my Internet was fine, and I deliberately disabled my network adaptor to see whether a loss of connection would be handled gracefully. It was not. Strictly speaking, is this not what the disconnect event is intended to handle? Its description is as follows: 'Emitted when the client's WebSocket disconnects'. Is this not a case of the client's WebSocket disconnecting?

it only emits when the socket disconnects in a way that it can't reconnect, usually we will automatically resume the ws for you.

@devsnek Okay. Well, this seems to be such a case, where the socket cannot reconnect. It would be nice for Discord.js to gracefully handle an exception such as this. It seems to fall within the scope of the event as described in the documentation, and would be very helpful in a library which handles WebSockets at its core.

@ChrisTalman How exactly would you handle such an exception? If you can't get DNS info about an address, then you can't reconnect. The only thing you can do at this point is to shut down the process.

@CharlotteDunois The context is that a network connection is not currently possible, but that it could be in the future. Thus, it is possible to reconnect, eventually. The library could handle this itself by reconnecting automatically, or application code could handle disconnect and call login at some interval.

This error happens when internet connectivity is lost by any means, not just manually disabling the NIC. Previously, the bot would sit there and wait and if the connection returned within about 2 minutes it would reconnect. Well, sometimes, it's always been flakey about reconnecting.
Anyway, the correct behavior would be to sit and wait, occasionally polling to see if it can reconnect. SAy once per minute or something to that effect. Not to just dump out and give up without the user's interaction.

I disagree, we aren't responsible for your internet. If you need to keep something hosted 24/7 buy yourself some hosting or get better internet.

That's not what the issue is about.

If that's the behaviour the user wants, I don't see how that's 'objectively stupid'. It's hardly like that would be an intensive operation, and it could be made optional. But, in any case, this issue started with the even more basic expectation that the disconnect event be fired in the case of a network failure instead of an exception being thrown. If this were done, a user could implement their own reconnection attempts. Although, having said this, having reconnection functionality out of the box would clearly be valuable to some users as has been demonstrated by this issue thread.

ChrisTalman: it used to have auto-reconnect. Prior to this error developing.

the option known as autoReconnect was removed in 11.x because we made it so you cannot turn it off. The issue with the error not bubbling up through client#onerror and client#ondisconnect will be fixed, but no new methods for "extended" reconnection will be added.

Personally, I think that the library should be tolerant of various network faults. If no one else bothers to implement this, I will eventually.

Mostly fixed in master, may still error on failed reconnects (not sure can't test atm)

This is not an issue on 11.3-dev, so I will close this.


fix(dag): replace non-existing wb/2022/wb_gender version by wb/2022-10-29/wb_gender

Hotfix for non-existing walden://wb/2022/wb_gender. @lucasrodes do you remember what happened here? Did you manually remove the old 2022 version from walden or was it something else that could have done it?

In this case, I disagree. We should commit to having a continuous, clean ETL run, and to fixing it within the hour if the build is failing. Otherwise, everyone else's work will get polluted with a spurious CI and confusing CI failure.


sl menu: There are no references to Ember.$, $ or jQuery

not ok 527 PhantomJS 1.9 - Unit | Component | sl menu: There are no references to Ember.$, $ or jQuery
    ---
        actual: >
            null
        message: >
            Died on test #1     at test (http://localhost:7357/assets/test-support.js:3025)
                at testWrapper (http://localhost:7357/assets/test-support.js:6192)
                at test (http://localhost:7357/assets/test-support.js:6205)
                at http://localhost:7357/assets/tests.js:25240
                at http://localhost:7357/assets/vendor.js:152
                at tryFinally (http://localhost:7357/assets/vendor.js:33)
                at http://localhost:7357/assets/vendor.js:158
                at http://localhost:7357/assets/test-loader.js:60
                at http://localhost:7357/assets/test-loader.js:51
                at http://localhost:7357/assets/test-loader.js:82
                at http://localhost:7357/assets/test-support.js:6024: Attempted to wrap $ which is already wrapped
        Log: |
    ...

Will likely be resolved by https://github.com/softlayer/sl-ember-components/issues/1503

Resolved by https://github.com/softlayer/sl-ember-components/issues/1503


BarChart doesn't response when all BarChartRodData.y = 0

I'm using Bart chart to display my monthly Spending vs Income. The chart works fine when there are at least 1 BarChartRodData.y (see my example code in the attachment) has none zero value, but if all BarChartRodData.y are zero, the app stop responding and crashed

I checked and it's working fine with 0 values on y.
Please make a reproducible code If you think anything is wrong.
Thanks!

It doesn't work for mine also.  It returns Another exception was thrown: Unsupported operation: Infinity or NaN toInt

Great!
Now you can provide me a reproducible code here.


[Event Hubs Client] Track Two: Fifth Preview (Client Builder Extensions)

Summary
Adjusting the extension methods in the ASP.NET client builder to make use of the new client type hierarchy.
Last Upstream Rebase
Wednesday, October 30, 3:02pm (EDT)
Related and Follow-Up Issues

Release Event Hubs Track 2 Library Preview 5 for .Net  (#8190)

I need to look at the processor host and see if we can add a method for it

I need to look at the processor host and see if we can add a method for it

It'd be reasonably easy to do now that the constructors have been mostly unified.  That said, I wonder if  it would fit in the type of hosts using the builder.  You really need a persistent host for the processor to be effective, since you can't influence when it pushes things at you.   That said, if you see a case for it, I'm happy to add a set of extensions for the next release.


Provide an example for NewTokenCredential

It would be very useful to see an example with the NewTokenCredential call. Is this used to authenticate using managed service identity? What should the value of Initial Token Value be?
Thanks!

Thanks @patnaikshekhar for reaching out!
The TokenCredential can be used for all scenarios. Are you particularly interested in MSI?
We can certain add a sample. I've logged this work item.

Yes a MSI example would be great. That is a pretty common use case. Thanks!

I am also trying to use NewTokenCredential to access azure blob with MSI.
@patnaikshekhar  where you able to achieve this?
@ zezha-msft  It would great to see a example on this.

Hi @VivekDevops, thanks for reaching out! We are about to make some major improvements in this area. I'll make sure that we include an appropriate sample this time.

Hi @zezha-msft thanks for the efforts. In the meantime, can you please give some guidance here.  I am trying to use NewTokenCredential to access azure blob.
I am able to generate the TokenCredential. However when I try to use the same to access the azblob, I am getting below errors.
===== RESPONSE ERROR (ServiceCode=) =====
Description=Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:de4bc8a4-a01e-0074-7259-27ab1d000000
Time:2020-05-11T06:00:20.0601875Z, Details:
Code: InvalidAuthenticationInfo
PUT https://xxxx.blob.core.windows.net/xxx?restype=container&timeout=61
Authorization: REDACTED
User-Agent: [Azure-Storage/0.7 (go1.14.2; linux)]
X-Ms-Client-Request-Id: [xx]
X-Ms-Version: [2018-11-09]
RESPONSE Status: 400 Authentication information is not given in the correct format. Check the value of Authorization header.
Content-Length: [298]
Content-Type: [application/xml]
Date: [Mon, 11 May 2020 06:00:19 GMT]
Server: [Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0]
X-Ms-Request-Id: [de4bc8a4-a01e-0074-7259-27ab1d000000]
The same token works, when I use the rest api instead of the azblob go sdk

Hi @VivekDevops, we have a test here that uses OAuth, perhaps it could help you?

Can we have an example how to use ManagedIdentity?


F/remove sklarn

removed dependency on Sklearn etc, in setup.py and all the places where I could find it
removed run() and other pieces of the API that aren't supposed to be used any more
made sure the tests cover the use cases
added notebook that showcases the use of the package (can be used as secondary testing)

Everything looks fine to me.
Albeit I couldn't run the notebook and test files since I don't have an account with a device.
But maybe that's something that is irrelevant to this PR.
@robinvanemden

You should definitely setup your own account and create a device so you can test!

On 10 May 2022, at 09:56, ayoub-assis @.***> wrote:
Everything looks fine to me.
Albeit I couldn't run the notebook and test files since I don't have an account with a device.
But maybe that's something that is irrelevant to this PR.
@robinvanemden https://github.com/robinvanemden
—
Reply to this email directly, view it on GitHub https://github.com/scailable/sclblpy/pull/49#issuecomment-1122054431, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGHUD44BM42NMX2YSK3TT3VJIJDTANCNFSM5VIG7IVA.
You are receiving this because you are subscribed to this thread.

The package documentation should be regenerated: main.m.html
This file is irrelevant now supported_models.json
docstring of `test_run_inputs'

Manual Testing

test_utils: passed

test_bundle: passed

test_main: passed

Notebook works fine

test_jwt: Passed after specifying the right usermanager server was correctly specified

Works for me locally. All issues that were found were also resolved by @ayoub-assis .
Merging.