Static website first load performance issue

Describe the bug
Static website first load performance issue
To Reproduce
Nodejs installs cyfs-sdk and cyfs-tool, publishes a web3 static web page, and it takes 80s to open the static page with a CYFS browser
1、Excute command: npm i cyfs-sdk -g and npm i cyfs-tool  -g
2、Use cyfs create -n my_web
3、Use cyfs deploy 200images  static web page
4、Use  CYFS browser  open the cyfs link to load  web page, it takes 80s load finisged
Expected behavior
CYFS Stack should optimize first load speed of web3 pages
System information
OOD ： linux <IP_ADDRESS>
CYFS browser: V <IP_ADDRESS> (beta)  windows
cyfs-sdk : 0.6.2

The full access link should be displayed, as the current cyfs supports three types of protocols: o, r, and a. Each protocol has different underlying implementations. Identifying which type of protocol is experiencing performance issues is essential for proper troubleshooting and optimization.

The full access link  link is protocol cyfs://a

I install this dec_app by cyfs://5r4MYfFYqjgMTa4JdYC4L87w6w4jUrQ1nUhYJ3AjcgfC/9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE.
And then open this web by link cyfs://a.3afs9n7yo81qswcb6yx02srtl4qlmumr2h3ys60bu4unil3o6p/9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE/index.html

I know that there are two ways to publish web pages in CYFS, cyfs deploy and cyfs upload, why there is a huge performance difference between the two ways

cyfs deploy, this method is to publish a dec app with service and web page at the same time, dec app can open dec app through the default cyfs://a link, and the page can interact with service at the same time, which is equivalent to the dynamic web page of web2,
Via cyfs deploy I deployed the dec app cyfs://5r4MYfFYqjgMTa4JdYC4L87w6w4jUrQ1nUhYJ3AjcgfC/9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE . Visit the webpage cyfs://a.3afs9n7yo81qswcb6yx02srtl4qlmumr2h3ys60bu4unil3o6p/9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE/index.html
The loading speed displayed by the browser is

200 requests
7.7 MB transferred
7.6 MB resources
Finish: 24.44 s
DOMContentLoaded: 809 ms
Load: 24.43 s

cyfs upload  , this method is to upload the webpage as a folder and generate the corresponding cyfs://o link, which is equivalent to the static webpage of web2.I published the same page cyfs://o/5r4MYfFYqjgMTa4JdYC4L87w6w4jUrQ1nUhYJ3AjcgfC/95RvaS5KmoTLU1KcxKeWUEHnbbVCm94nSNPVbvYYt9Cq/index.html
The loading speed displayed by the browser is

199 requests
7.7 MB transferred
7.6 MB resources
Finish: 528 ms
DOMContentLoaded: 93 ms
Load: 701 ms

In the request from CYFS runtime to OOD gateway, both the a protocol and the o protocol access files by downloading the corresponding file chunks in the final step. Therefore, both protocols are affected by the issue of "chunks not being cached correctly." After fixing this bug, you can try the mentioned tests again to see if there is an improvement in performance.
See at #149

When opening an "a" link in cyfs-browser, what happens?
For example:
cyfs://a.3afs9n7yo81qswcb6yx02srtl4qlmumr2h3ys60bu4unil3o6p/9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE/index.html
In the cyfs-runtime protocol stack of cyfs-browser, a standard process will take place as follows:
1. Parse the "a" link to find the target DEC-ID and the content to access in the DEC.
For the given URL, the parsed results are:

DEC-ID
dec-id is 9tGpLNngnJPovaM2vBouhFSeCctAboE3oinaE8L7MCmE
index.html
directly specifies the inner_path, indicating the content of the currently installed version (since the URL does not specify a version) of the DEC's built-in page: index.html
Each DEC, after installation, can specify a web page directory, which is stored in OOD as an objectmap (dir).

2. Find the dir-id of the current version of the target DEC.
This requires sending a query request to OOD to find the dir-id of the built-in web page for the current version of the DEC, which is a time-consuming operation.
3. If the DIR-ID is found in step 2, the URL request is converted into an "o" link:
cyfs://o/{ood-id}/{dir-id}/index.html
(Where the OOD-ID is the default target for the "a" link, i.e., the current zone's OOD)
Then continue with the subsequent processing steps.
4. For the "o" link in step 3, the target object needs to be acquired first.
The cyfs-runtime directly sends a non.get_object request to OOD (the default target for the "a" link) to obtain the object corresponding to {dir-id}/index.html, which is also a time-consuming operation. After obtaining the target object, usually a FileObject, the following steps are carried out.
5. For the target object (FileObject) obtained in step 4, download the corresponding data.
The cyfs-runtime directly sends an ndn.get_data request to OOD to obtain the binary data corresponding to the target object, i.e., the binary content of index.html. This step is also a time-consuming operation.
Some conclusions
As can be seen from the above steps, when cyfs-runtime opens a built-in page of a DEC through an "a" link, each FileObject requires a complete five-step process, with three steps requiring interaction with OOD, i.e., there are three time-consuming operations that need to be optimized.

step 5 #149 issue I verified successfully, but cyfs://a link may still consume a lot of time in step 3 and step 4, resulting in slower page loading equivalent to web2 web pages
cyfs-runtime.zip

step 5 #149 issue I verified successfully, but cyfs://a link may still consume a lot of time in step 3 and step 4, resulting in slower page loading equivalent to web2 web pages cyfs-runtime.zip

Thanks for the test cases and logs you've provided!
From the logs, it appears that the cache mechanisms for step 3 and step 4 are effective when opening the "a" link for the second time.
1. When opening the "a" link for the first time
the key logs for step 3 and step 4 are as follows:
Details

[2023-04-06 14:23:52.149996 +08:00] INFO [ThreadId(14)] [component\cyfs-stack\src\front\service.rs:50] will process o request: FrontORequest { source: protocol=http-local, zone=(CurrentDevice-Some(5aSixgNR5psWnP2wiZp32nDFCYUEpA4knABQLBj3GAgM)-Some(5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1)), dec=9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG, verified=None, req_path: None, target: [5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7], object_id: 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, inner_path: Some("/index.html"), range: None, referer_objects: [], context: None, group: None, mode: Default, format: Default, flags: 0 }
[2023-04-06 14:23:52.156453 +08:00] WARN [ThreadId(14)] [component\cyfs-stack\src\non_api\inner_path\objectmap_loader.rs:45] get_object with objectmap and inner_path but not found! object=95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, inner_path=/index.html
[2023-04-06 14:23:52.156593 +08:00] WARN [ThreadId(14)] [component\cyfs-stack\src\non_api\inner_path\service.rs:164] get object with inner path but not found with missing cache! object=95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, inner_path=/index.html
[2023-04-06 14:23:52.458361 +08:00] INFO [ThreadId(9)] [component\cyfs-lib\src\requestor\bdt.rs:169] http-bdt request to 5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7:84 success! during=301ms, seq=TempSeq(4185446238)
[2023-04-06 14:23:52.458842 +08:00] INFO [ThreadId(9)] [component\cyfs-lib\src\non\requestor.rs:294] get object from non service success: 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB/index.html, object=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp

From the logs mentioned above, it is clear that due to the lack of local cache, a get request was sent to the target OOD(5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7), and the target File object(7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp) was successfully obtained.
2. After successfully opening the link, the cyfs-runtime updates the local cache
the key logs are as follows:
Details

[2023-04-06 14:23:52.475155 +08:00] INFO [ThreadId(14)] [component\cyfs-noc\src\relation\sqlite\db.rs:159] insert object relation success: key=NamedObjectRelationCacheKey { object_id: 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, relation_type: InnerPath, relation: "/index.html" }, target=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp
[2023-04-06 14:23:52.477838 +08:00] INFO [ThreadId(9)] [component\cyfs-noc\src\meta\sqlite\db.rs:468] insert new to noc success: obj=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp, access=rwxrwxrwx---rwxrwx
[2023-04-06 14:23:52.478000 +08:00] INFO [ThreadId(9)] [component\cyfs-noc\src\storage\local.rs:56] meta put object success! request=source=protocol=http-local, zone=(CurrentDevice-Some(5aSixgNR5psWnP2wiZp32nDFCYUEpA4knABQLBj3GAgM)-Some(5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1)), dec=9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG, verified=None, object=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp, storage_category=Storage, access=258559, insert_time=13325235832458990, owner=5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1, object_update_time=13325229952540771, ret=result=Accept, object_update_time=13325229952540771
[2023-04-06 14:23:52.484537 +08:00] INFO [ThreadId(9)] [component\cyfs-stack\src\non_api\noc\processor.rs:117] put object to local noc success: id=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp, access=None

From the logs mentioned above, it is clear that two cache were updated:

relatetion cache
(95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB/index.html) -> 7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp

noc cache
The target object has been save to local's noc: 7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp

3. During the second time the "a" link is opened, step 3 and step 4 directly retrieve the target File object from the local relation cache and noc cache, without sending another get_object request to the OOD
The key logs for this process are as follows:
Details

[2023-04-06 14:24:56.268332 +08:00] INFO [ThreadId(2)] [component\cyfs-stack\src\front\service.rs:50] will process o request: FrontORequest { source: protocol=http-local, zone=(CurrentDevice-Some(5aSixgNR5psWnP2wiZp32nDFCYUEpA4knABQLBj3GAgM)-Some(5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1)), dec=9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG, verified=None, req_path: None, target: [5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7], object_id: 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, inner_path: Some("/index.html"), range: None, referer_objects: [], context: None, group: None, mode: Default, format: Default, flags: 0 }
[2023-04-06 14:24:56.268558 +08:00] INFO [ThreadId(2)] [component\cyfs-stack\src\non_api\router\router.rs:397] router get_object default handler, req=common: req_path: None, protocol=http-local, zone=(CurrentDevice-Some(5aSixgNR5psWnP2wiZp32nDFCYUEpA4knABQLBj3GAgM)-Some(5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1)), dec=9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG, verified=None, level: router, target: 5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7, flags: 0, object_id: 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, inner_path: Some("/index.html"), router=protocol=http-local, zone=(CurrentDevice-Some(5aSixgNR5psWnP2wiZp32nDFCYUEpA4knABQLBj3GAgM)-Some(5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1)), dec=9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG, verified=None, target: 5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7, direction: local_to_local, next_hop: 5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7, next_direction: local_to_local
[2023-04-06 14:24:56.288849 +08:00] INFO [ThreadId(2)] [component\cyfs-noc\src\meta\sqlite\db.rs:844] noc meta update last access success: obj=7Tk94YfH6YtFR3RReU8Z2PJww879fVESXtwq8gZHXYQp, last_access_time=13325235896279233, last_access_rpath=None
[2023-04-06 14:24:56.289657 +08:00] INFO [ThreadId(9)] [component\cyfs-stack\src\non_api\router\router.rs:435] router get_object from local noc cache! id=95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, next_hop=5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7

The logs here for requests with inner_path are somewhat ambiguous and need to be improved.
"router get_object from local noc cache! id=95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB, next_hop=5aSixgM3hFBdnefL7W67Gemq8XJzwxJscpTLF17JpdP7"
In fact, the local query found the target file corresponding to 95RvaS5GZN8jwdx37hddykZYSbuFJvn8KTL6N1Scp3MB/index.html.

The optimization based on cache has been added to Step 1 of the a-link process.
See at https://github.com/buckyos/CYFS/issues/179#issuecomment-1499949023
On this basis, the performance can be retested @lizhihongTest

Thank you very much for your answer.
A-link and O-link of the CYFS protocol, in my opinion, currently satisfy users' performance needs for webpage loading.
Test Environment
In this test, I added an HTTP control group, and provided access to the Dec app web page in my VPS OOD through Nginx proxy for HTTP access.

OOD Servcie : Nightly <IP_ADDRESS>4
CYFS Browser : Nightly  build by CYFS main branche lastest code 1fb3857
web3 cyfs://o link : 200 images web dir cyfs://o/5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1/95RvaS5Wx4Ai4xTqBsceFE7z1GFNX8GZohiX59Pxmk7n/index.html
web3 cyfs://a link : 200 images web,dec_app cyfs://5r4MYfFJ7ktzBzSi1sWmU7BJLgpEEdp8ukbWemsFuQc1/9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG ,web page :  cyfs://a.3afs9n7yaxaxbxzw5l7aum09336dambv92ga2q935uairdhkc9/9tGpLNnCgANJ6PfqfQSAvLu2tVv4qLG1cFoTgZBEkNcG/index.html
web2 http link : 200 images web ,Nginx proxy OOD`s static web ：http://<IP_ADDRESS>/

Test Result
cyfs://o link

First load time

199 requests
7.7 MB transferred
7.6 MB resources
Finish: 17.31 s
DOMContentLoaded: 585 ms
Load: 17.31 s

Clear browser cache, second load time
200 requests
7.8 MB transferred
7.7 MB resources
Finish: 3.82 s
DOMContentLoaded: 125 ms
Load: 3.81 s

cyfs://a link

First load time

200 requests
7.8 MB transferred
7.7 MB resources
Finish: 18.31 s
DOMContentLoaded: 1.52 s
Load: 18.31 s

Clear browser cache, second load time

200 requests
7.8 MB transferred
7.7 MB resources
Finish: 3.82 s
DOMContentLoaded: 125 ms
Load: 3.81 s

HTTP link

First load time

201 requests
8.5 MB transferred
8.5 MB resources
Finish: 9.75 s
DOMContentLoaded: 250 ms
Load: 9.69 s

Clear browser cache, second load time

201 requests
8.5 MB transferred
8.5 MB resources
Finish: 8.12 s
DOMContentLoaded: 511 ms
Load: 8.08 s

Thoughts on test results
During the initial loading of resources, CYFS may be slower than HTTP, but this is mainly due to protocol differences.
+ CYFS divides files into File Object and chunk raw data for data confirmation, resulting in a larger amount of data being transmitted than traditional HTTP protocol during webpage loading.
+ Additionally, the BDT transmission protocol encrypts the transmitted content, which takes some time.
+ The NDN algorithm used is a multi-source download algorithm, and performance can be further improved in the case of multiple sources.
The speed of loading the same resources in CYFS is faster than HTTP. This is mainly due to CYFS's secondary caching in the device disk, which is under the browser cache. This mechanism may have the following advantages and disadvantages:
+ Users may experience faster loading speeds when opening the same file.
+ Due to the unique mapping relationship between CYFS O-link and files, CYFS's disk cache is also effective in cross-DEC APP scenarios.
+ There is currently no recycling mechanism for CYFS Stack to cache data, which may consume a large amount of system disk space.


[Please disregard] Looking for guidance troubleshooting freeze on start

Issue Report Checklist

[x] Searched the issues page for similar reports
[x] Read the relevant sections of the Spyder Troubleshooting Guide and followed its advice
[ ] Reproduced the issue after updating with conda update spyder (or pip, if not using Anaconda)
[x] Could not reproduce inside jupyter qtconsole (if console-related)
[x] Tried basic troubleshooting (if a bug/error)

[x] Restarted Spyder
[x] Reset preferences with spyder --reset
[ ] Reinstalled the latest version of Anaconda
[x] Tried the other applicable steps from the Troubleshooting Guide

[x] Completed the Problem Description, Steps to Reproduce and Version sections below

Problem Description
Spyder freezes immediately after starting. All widgets load (Editor, Console, Help, other tabs are visible), cursor blinks in the editor, then after maybe 1 or 2 seconds the application doesn't respond (and cursor stops blinking).
The widgets are actually processing keystrokes during the 1 or 2 seconds before freeze (so if I do quickly I can send input to the editor, the console, and also switch to the history tab which shows it is loaded).
What steps reproduce the problem?
pacman -S spyder for the Arch package version, or git clone and bootstrap.py for the git checkout; I tried only with latest version on 5.x branch, so it's 5.3 for the git checkout while the Arch package is 5.2. Both behave exactly the same, including debug log and gdb backtrace.
Body of my issue
I know this is not an Anaconda setup and there's no official support, and I would like foremost to ask for guidance in troubleshooting the problem.
What I was able to get so far is a debug log and a backtrace. I did with both Arch 5.2 and git 5.3, from what I can see the result is the same. I assume the results from the git checkout will be most useful:
Spyder log: debug-checkout.log
gdb py-bt output: pybt-checkout.log
gdb raw bt output: raw-bt-checkout.log (this is from a different session, but it should match closely, the thread numbers are always the same throughout my tests from what I can see)
This is from python bootstrap.py -- --safe-mode --debug-info verbose. The py-bt log shows a thread apply x-y-z py-bt followed by t a x-y-z py-list, where x-y-z is my selection of threads (those that don't say Unable to locate python frame).
The Spyder log looks ok to me, I compared with a debug log from a working Spyder on a Windows machine, I don't see any difference, including to the point reached before starting interaction.
From there I don't know how to troubleshoot further. My guess is that I should concentrate on thread 1, but it's no more than a hunch. It spends most of its time in QApplication::compressEvent from libQt5Widgets.so.5, which may just mean that Qt is gently sitting in its event loop? I think I caught a glimpse of an infinite recursion error while trying to step through, but I'm really not sure.
So, I'd be glad to try to follow any advice for pursuing troubleshooting, which I guess would start with finding more useful logs (possibly installing more debug symbols so that these backtraces look more human)?
Other notes

pylsp gets loaded, often with a delay of ~20 seconds (but sometimes it's very quick, couldn't discern a pattern); its status then gets updated in the status bar even though Spyder is not responding. pylsp itself works.
jupyter qtconsole is working beautifully, including "inline help" (not sure what it's called).
I tried the PySide2 library, it gives the same result, including pretty much the same gdb backtrace.
I tried to run runtests.py from the git checkout; it gave a lot of errors (as well as many successful tests) and eventually crashed with a segmentation fault in test_ipythonconsole.py:
spyder-runtests.log

Versions

Spyder version: 5.2.0 (Arch package), 5.3.0.dev0 rev 9a0cab850 (git checkout)
Python version: 3.9.9
Qt version: 5.15.2
PyQt version: 5.15.6
Operating System name/version: Arch Linux

Dependencies
Cannot get from Spyder, listing the ones that (I'm guessing) could be relevant:
IPython: 7.30.1
qtconsole: 5.2.1
qtpy: 1.11.2
spyder-kernels: 2.2.0
lsp-server: 1.3.2

Getting some more advice, I tried with another user, and then Spyder is running fine. So the issue is mostly irrelevant.
How can I try to identify the problem with my current user though?

Short of being able to delete the issue, I understand I should pinpoint the user issue. So I'll try to do that and open another issue if needed.
Pleade disregard this issue.

I had the same problem, also on Arch using the official repository package. I ran it under a new user and it worked. In my case I think the issue was kvantum themes. Going back to default Plasma setup resolved the issue on my user as well.

Yes, it's related to kvantum, the way qtconsole (the Spyder console) handles its text area widget leading to a loop of resize events that makes the application unresponsive.
My current workaround is alias spyder='QT_STYLE_OVERRIDE=Fusion spyder'
The issue is tracked at #16826.


Add curator

This pull request adds curator (running in a docker container as all other parts of the ELK stack).
The image allows for following environment variables:

elasticsearch host (ELASTICSEARCH_HOST)
cron configuration (CRON)
curator config file (CONFIG_FILE)
curator action file (COMMAND)
curator filtertype age unit count (UNIT_COUNT)

By the way, why not using the Dockerfile provided by the curator project?

Thanks for sharing your work :), happy to contribute.


Monitor user application availability during chaos

Description
Current Kraken integration with Cerberus monitors the cluster as well
as the application health post chaos and pass/fails if they are not
healthy after chaos. This commit adds ability to monitor the user
application health during the chaos and fails the run in case of
downtime as it's potentially a downtime in case of customers environment
as well. It is especially useful in case of control plane failure
scenarios including API server, Etcd, Ingress etc.

Can one of the admins verify this patch?

When running the get_status function at the end of each of the scenarios it exits if the cerberus status is false. In this case the cerberus status could be false because we are checking for a failed route, so it never gets to the application_status function. I think we need to take out the sys.exit(1) here so that we can continue to check the application url

When setting cerberus to true and monitor routes to false I get the following error:
UnboundLocalError: local variable 'application_routes_status' referenced before assignment

When setting cerberus to true and monitor routes to false I get the following error:
UnboundLocalError: local variable 'application_routes_status' referenced before assignment

Updated the PR to fix it. PTAL. Thanks.


Feature Request: Upload changed files.

I was thinking about this earlier, and I think I have a solution. Ruby isn't my language, but I could try (in a bit) to execute. Anyway, I was thinking, first upload with glynn, a file would be created containing a modified date (or an MD5) for each file. This file would also be on the server. When glynn would run, it would check if any MD5s changed, and upload those corresponding files. Just thinking logically, and programmatically, it would go something like this:
glynn upload command
-> check MD5 file (for loop through files I guess)
--> has file changed?
--->Yes? Upload file, exit for loop
---> No? Exit for loop.
--> end for loop

This is definitely the right way of doing this.

@dmathieu wow! quick response! I'd love to try, but Ruby, as I said, is not something I have good understanding of.

Unfortunately, I haven't used glynn in years, and don't have any FTP server available at hand to test this.
I might be able to implement this in the coming days/weeks, but will need someone to make sure it does properly work.

@dmathieu I'd happily help. I started something (from some internet examples)
Dir.foreach(ftp_dir) do |item|
  next if item == '.' or item == '..'
  File.write('md5', Digest::MD5.hexdigest File.read item)
end

Ok, I've been messing around a bit, and I've got this far:
https://github.com/128keaton/glynn/blob/master/lib/glynn/ftp.rb#L80
I am, however, having issues. I can read the digest of the MD5s of things inside of folders, but they don't want to write for some reason..

I was unable to make progress, any word from anyone else?


API proposal to get the computed measure size of a node

API proposal
I am using the Javascript Yoga npm package and have an API proposal to get the measure size of a node. Similar to the getComputedLayout() method, I am proposing that a getComputedMeasureSize() method is added. Or, perhaps the measure size be included in getComputedLayout() so that the returned object looks something like { left: 0, right: 0, top: 0, bottom: 0, width: 0, height: 0, measureWidth: 0, widthMeasureMode: 0, measureHeight: 0, heightMeasureMode: 0 }. I'm not familiar enough with the source code to know if this makes sense for every node. I am assuming that every node has a measure size, but it is only exposed to the developer when a measure func is invoked.
My use case
I am integrating Yoga with a custom 3D layout system and it is very important that I compute a measure size for Yoga nodes, particularly leaf nodes with a measure func, so that I can compute a measure size that also includes depth using additional custom logic.
My attempt at a solution with the current API
If Yoga executes the measure func, the measure size is provided as parameters to the measure func, so it can easily be used. But, if the measure func is not invoked because the leaf node has fixed dimensions, I need to compute a measure size by basically duplicating what I think Yoga does to compute a measure size which of course is very hacky and bug-prone. This is complicated by the fact that Yoga uses caching. So, after the first execution, it's difficult to know whether Yoga is still using the measure func result, or things have changed such that the node is no longer auto sized. In both cases, the measure func won't be executed, so I don't know whether I should use the measure size of the last invocation of the measure func, or compute a measure size myself.

If Yoga executes the measure func, the measure size is provided as parameters to the measure func, so it can easily be used. But, if the measure func is not invoked because the leaf node has fixed dimensions, I need to compute a measure size by basically duplicating what I think Yoga does to compute a measure size which of course is very hacky and bug-prone.

As you mention, Yoga is not guaranteed to call the measure function, and the last call might not be the final layout. As part of YGNodeCalculateLayout, Yoga will set a flag on every Node where it or one of its children may have a new layout result compared to the last layout. A common pattern is to traverse down, applying the results of any nodes with a new layout, and resetting the flag.
The size which should be imposed on the content is the computed layout of the node, minus any padding or border. Yoga will have called into the platforms measure function as part of determining this.
There's an open issue where JS does not currently expose YGNodeGetHasNewLayout or YGNodeSetHasNewLayout but I think adding it would be pretty simple. https://github.com/facebook/yoga/issues/681

I'm not sure that exposing YGNodeGetHasNewLayout would solve my issue.
Let me try to articulate my problem a bit more in a visual way.
Below is a screenshot where the gray node is the Yoga root node, and the red node is an auto-sized leaf node where the measure width comes from the root node. The white squares are arbitrary children that can be thought of as objects outside of the Yoga layout.
The leaf node distributes N children in an ellipse that is the size of the available area. Notice that the distribution is different depending on the available size, so the results are different if the available width is 10 vs. 3, for example. No matter if the leaf node is auto or fixed size, an algorithm needs to execute to distribute the children after Yoga has processed. It's crucial that the children are distributed against the measure size, not the computed size. For example, using the screenshot as a reference, if the width measure size of the leaf node is 10, the computed width might only be 2. It would be incorrect to distribute the children in the leaf node using a width of 2 because it changes the shape and size of the distribution - it would be much more narrow than it should be.

So, things are trivial on the first execution of Yoga. I know if the measure func has executed and I have the measure size. If the measure func didn't execute, I can compute a measure size using the node's padding and borders.
But, after that, it seems nearly impossible to get a reliable measure size because of caching. Even if I had a YGNodeGetHasNewLayout value, it wouldn't tell me if the node is auto-sized or fixed size, and which dimensions specifically are auto/fixed. I also want to point out how difficult it would be to determine if a dimension is auto or fixed. Even if a dimension is set to 'auto', it may have a 'flex' value that effectively makes it fixed-size. Further, whether a dimension is auto-sized is dependent on the parent flex direction and align type. I'm sure there are other things I'm not even thinking of.
Possible Yoga API solutions:

I think the ideal solution would be the ability to get the measure size of a node at any time after Yoga has executed. I know the measure func can be executed more than once, but after execution, isn't there a final measure size that has been computed/used for a node?
Some method to determine if the measure func was used during the last execution of Yoga, even if it was not invoked because of caching. This allows the developer to know if the last provided measure size should be used, or all dimensions are fixed and a measure size can be computed using padding/borders.
Some method to determine if a dimension is auto-sized / measured using the logic described above. If either the width or height are auto, it can be assumed that the measure func is being used, and the last provided measure size can be used. Otherwise, the node should be treated as fixed-size.

Or, another way to say it - isn't there a measure size that corresponds to the computed size of a node? Also, if the node is fixed-size, then the measure size would be the computed dimensions minus padding and borders, and both the width/height measure mode would be 'fixed'.

The content size should be the final computed size (border-box size), minus padding and borders, regardless of the mode of measurement. I.e. you should read this off of the layout results to use for sizing with the external layout system.
The results from a measure function should not be assumed to be related to the final layout of the node.

Okay, I think that's my disconnect. I don't think my use case is supported by Yoga. Can you confirm Nick, and do you understand what I'm trying to articulate with my description and image? It's sort of a catch-22. If Yoga asks me the size of my node, I need to use its provided measure size to distribute my items and return a computed size. But, I can't then later use that computed size to distribute my items because that makes a distribution that has a different size than what was provided to Yoga in the measure func result and the layout is simply incorrect.
If this use case isn't supported by Yoga, do you have any other suggestions on how I might at least partially implement this? Identifying if a dimension is auto-sized would help, but as I mentioned in my previous message, I feel like that might be too complicated. Also knowing whether the measure func is being used (even if cached) would help, but I've run out of ideas on how to reliably know that.

If I am understanding correctly, the overall scenario should be supported, but how it needs to be handles depends on what the hosting code looks like.
The measure function is Yoga asking for hypothetical size (but Yoga can impose its own).
After the Yoga layout step, the leaf node with measure function has a known size, as calculated by Yoga, but the item has not yet been laid out.
The step needed, is after Yoga does layout, the node with measure function should be told to calculate its own layout, using the final result Yoga came up with. I.e. tell the red box to distribute its items, using the final layout result from Yoga.
This will incorporate intrinsic size, but might be larger, e.g. if user sets explicit width/height. How to handle the larger box than natural content depends on desired integration with external layout system.

Hmm, I'm still not seeing how my use case would be possible.
Here's another screenshot of the entire layout process. To summarize, the leaf node distributes N children in an ellipse that is the size of the available area.

The crux of the issue here is that the distribution algorithm is not linear like traditional layout systems. Notice that if I were distributing items against a horizontal line, like Yoga, this issue would not exist.

Okay, when you mention "measure size", you mean the available space the node can be sized into. I think I have been misunderstanding that.
This size is not always consistent with the parent. E.g. Yoga (or other layout systems) can size the node into infinite available space (like when overflow: "scroll" is set). Or, the parent may be sized on its content, and its final dimensions end up smaller than the available size.
In this example, it seems like you could distribute using the measured size of the parent. Is that something you could do more generally?

Ah, sorry. When I say "measure size" I'm referring to the parameters of the measure func (width, widthMode, height, heightMode). From experimentation, it seems that the measure size of an auto dimension comes from the first fixed-size ancestor. The measure size of a fixed dimension is the border box size (the fixed dimension minus border and padding).

In this example, it seems like you could distribute using the measured size of the parent. Is that something you could do more generally?

That seems intuitive but I'm not sure that would work in all cases. What if the parent is autosized? Then the measure size comes from an ancestor. The key to the issue is knowing what size was used to produce the measurement (the parameters of the measure func). I've explained above why that is difficult to get reliably because of caching, but let me know if you want me to expand on that.

Just wanted to share my tentative solution. When the measure func is invoked, I am saving both the measure size (measure func params) and the measurement result. In my custom distribution phase, if the computed size of the node equals the saved measurement result, I am assuming the node is being measured and distributing against the saved measure size. I don't think that's perfect but should work reasonably well.
Nick, if you're not interested in any of my API proposals, you can go ahead and close the issue. I'll also put in a vote for exposing the YGNodeGetHasNewLayout like you originally mentioned. That would be useful to my project in multiple ways.


Enhancements for working with basic auth enabled Solr clusters

For background on Solr basic auth, see:
https://lucene.apache.org/solr/guide/8_8/basic-authentication-plugin.html#basic-authentication-plugin
The Solr operator should provide an easy way to enable basic authentication for a SolrCloud cluster.
There are two main uses cases we should support:

The operator should bootstrap a default security.json with basic auth enabled. This allows users to easily spin-up a secure Solr cluster without having to know all the details of how Solr security works. Having the Solr API be protected by basic-auth out of the box leads to a better getting started experience for Solr and the operator in general.
The operator should allow users to supply their own security.json (via ConfigMap) and corresponding secret with credentials the operator should use to make API requests to Solr. Both are needed and if the user chooses to supply their own security.json then they need to make sure the credentials are in the secret as well. For this use case, the operator's main job is to bootstrap the security.json in ZK as part of the initialization process.

When basic auth is enabled, we need to ensure the liveness / readiness probes work (/admin/info/system endpoint). In addition, the rolling restart logic requires access to the /admin/collections endpoint. Also, backup / restore requires API access. Thus, the Solr operator needs access to a username and password to make API calls. These credentials will come from a Kubernetes secret in the same namespace.
For the first use case (bootstrap a default config), the only tricky part is we need to create the user password hashes in the same manner that Solr expects them. The default security.json will define an admin user, a general purpose read-only solr user, and special k8s-oper user that only has read access to the API endpoints needed to operator in Kubernetes, such as the /admin/info/system endpoint used by probes. The password for each user can be retrieved using kubectl using:
kubectl get secret dev-solrcloud-basic-auth -o jsonpath='{.data.k8s-oper}' | base64 --decode

Also, I'm not sure if the operator should watch the security.json ConfigMap for updates (as we do with other user-provided ConfigMaps) and try to apply the updated JSON? There's an API for updating the security config and the ref guide suggests using that so maybe the operator just bootstraps the file and then it's up to users to maintain via the API? Trying to accommodate user updates via the API and/or ConfigMap could get messy.
For the probes, K8s allows specifying a header for HTTP endpoints, but then our password will live in the STS config. I think it would be better to set an env var from the secret and then run a command for the probe using this env var. The SolrCLI command-line tool provides an api action that we can use for the probes:
java -Dbasicauth=k8s-oper:****** \
  -Dsolr.httpclient.builder.factory=org.apache.solr.client.solrj.impl.PreemptiveBasicAuthClientBuilderFactory \
  -Dsolr.install.dir="/opt/solr" \
  -Dlog4j.configurationFile="/opt/solr/server/resources/log4j2-console.xml" \
  -classpath "/opt/solr/server/solr-webapp/webapp/WEB-INF/lib/*:/opt/solr/server/lib/ext/*:/opt/solr/server/lib/*" \
  org.apache.solr.util.SolrCLI api -get http://localhost:8983/solr/admin/info/system

Unfortunately, bin/solr does not expose the api action :-( So we'll need to take some liberties on how the probe command is constructed, such as assuming Solr is installed at /opt/solr
Still testing / tweaking the default security.json but I imagine it would be something like this:
{
      "authentication":{
        "blockUnknown": true,
        "class":"solr.BasicAuthPlugin",
        "credentials": {
          "admin":  "...",
          "k8s-oper":     "...",
          "solr":   "..."
        },
        "realm":"Solr Basic Auth",
        "forwardCredentials": false
      },
      "authorization": {
         "class": "solr.RuleBasedAuthorizationPlugin",
         "user-role": {
            "admin": ["admin", "probe"],
            "k8s-oper": ["probe"],
            "solr": ["users", "probe"]
         },
         "permissions": [
           { "name": "k8s-probes", "role":"probe", "collection": null, "path":"/admin/info/system" },
           { "name": "k8s-status", "role":"probe", "collection": null, "path":"/admin/collections" },
           { "name": "all", "role":["admin","users"] },
           { "name": "read", "role":["admin","users"] },
           { "name": "update", "role":["admin"] },
           { "name": "security-read", "role": "admin"},
           { "name": "security-edit", "role": "admin"}
         ]
      }
    }

Just having #1 done for now sounds like a good place to be in. #2 adds complexity and requires the operator to deal with more than what the users might want currently.


Azure pane authentication problems

This fixed my SignIn Problems (even with different Tenants), i can now work on the Azure Databases.
But if I i right click on Database to Manage in the Azure Pane, then i can not connect because the connection will not be pre configured with the right Authentication Values. I have to select Azure AD MFA as Authentication, then it works.
Originally posted by @DerAlbertCom in https://github.com/microsoft/azuredatastudio/issues/7619#issuecomment-570784737
We need to opportunistically try to use AAD with Azure Pane, and if that doesn't work silently fallback to SqlLogin for users.

Fixed :)

Fixed :)

Which version?
Version: 1.16.1
Commit: 16801c522df68e63218b39ccb5b4a7431f9bf667
Date: 2020-03-23T22:06:16.901Z
VS Code: 1.42.0
Electron: 7.1.11
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.4.0

stil a problem (freshly updated)

Nope. Not fixed, I've updated to 1.17
I had to reconnect to Azure (that's may be because of update azure handling, so no bug there).
I can list all the Database and Servers in the Azure Pane, but If I wan't to connect i still have to change Authentication Type and Azure AD Tenant.
If i try direct to Manged a database then I get the login error for user INSERTSERVERADMINUSERNAME.

That's expected behavior at the moment. I'll make an issue with this feature request.

Unfortunately we had to revert this change at the last minute as it was causing other issues :(
I'll reopen this and start tracking it again.

Any update on this?

Issue is now fixed as users can choose their default auth type for Azure resources with below setting:


Allow ingressClassName to be unset

Signed-off-by: Craig Jellick<EMAIL_ADDRESS>

addresses https://github.com/acorn-io/acorn/issues/493


New package: Fibers v1.0.0

Registering package: Fibers
Repository: https://github.com/lincbrain/Fibers.jl
Created by: @ayendiki
Version: v1.0.0
Commit: 2d980e6858902d0a2d5ca2048fe38d04171a7227
Reviewed by: @ayendiki
Reference: https://github.com/lincbrain/Fibers.jl/issues/17
Description: Julia package for reconstructing fiber trajectories from multi-scale, multi-modal imaging data

[noblock] name similarity is okay, but the other issues will have to be addressed


package version not being exported at build time

v0.2.0’s __version__ exported as ”(unknown)” for some reason. I should probably just be using a proper poetry plugin for this one.

https://pypi.org/project/poetry-version-plugin/

looks like poetry honors .gitignore, oops. https://stackoverflow.com/questions/66313531/how-to-prevent-poetry-to-consider-gitignore


[Set Up Rust] Example code does not cause a SSE error anymore

See http://os.phil-opp.com/set-up-rust.html#comment-2425734866

This code seems to still trigger the error:
let mut a = ("hello", 42);
a.1 += 1;


Building a BoundingFrustum from an orthographic projection has errors

When creating a BoundingFrustum from an orthographic projection, the following values are created for the slopes
RightSlope = -0.5
LeftSlope = 0.5
TopSlope = -0.5
BottomSlope = 0.5
The correct values for the slopes for an orthographic projection should all be 0.  There is no slope to the planes in an ortho projection.
The error is caused by the assumption that the x/y values will vary from 0 to the provided value over the z range.  But, in ortho x/y will not vary over the z range.
If you create a Frustum like this, all collision tests will be wrong, since the wrong planes will be used for testing.
This could be fixed by using all 8 points of the frustum instead of just 4.

At further looking, seems that BoundingFrustum is not really adaptable to ortho, since it doesnt store the plane offsets.  Perhaps the documentation should be updated to reflect that it doesnt function with ortho at all.

A Frustum is almost always associated with a perspective projection AFAIK for collision volumes. Wouldn't an orthographic projection just be a bounding box or oriented bounding box?

Fixed in this PR on Microsoft Docs.


Better FormattedCodeblock fontsize adjustment

FormattedCodeblock does not line wrap text like FormattedLabel, but instead down scales the font size such that the content fits into the bounding box.
The current strategy to finding the right font size is to iteratively layout the text and check whether it fits into the given bounding box. If not, decrease the font size by one and try again till it fits.
I already noticed some lag whenever this iteration kicks in. Perhaps we can speed it up a bit by providing a better guess for the font size if the first iteration failed. E.g. use an average font size estimated as (current font size) * (current text width) / (bounding box width).

#26 implemented an alternative strategy, e.g. keep the previous text width around and whenever we have to re-layout the text then we start the iteration from there. Let's see how good this works.


You are trying to sign in from a browser or app that doesn't allow us to keep your account secure

PLEASE READ THE POSTING GUIDELINES AND ANSWER THE QUESTION BEFORE POSTING, OTHERWISE ISSUE WILL BE CLOSED AND MARKED AS INVALID

I hereby declare the following issue is a [tool specific question/bug report] and it is NOT a help request about creating a phishlet.
I am fully aware that this is not a customer support portal, I can't demand answers and I'm aware I am using a free tool.
I am not going to use Evilginx to hax my girlfriend's account or use it for any other illegal purpose.
I am not trying to set up a domain on FreeNOM (also read the sentence above again).
I am not a robot.

Please type in "I CONFIRM" below if you confirm the sentences above or otherwise make some funny remark:
When i test it the site says "You are trying to sign in from a browser or app that doesn't allow us to keep your account secure"
Thanks!

Anybody find any solution for this?


Firefox tests on MacOS are much slower compared to Ubuntu tests

Examples:
firefox-headless tests on macos-latest
succeeded 2 hours ago in 27m 25s
https://github.com/puppeteer/puppeteer/actions/runs/4142214498/jobs/7165355344
firefox-headless tests on ubuntu-latest
succeeded 1 hour ago in 13m 7s
https://github.com/puppeteer/puppeteer/actions/runs/4142214498/jobs/7165354734

cc @whimboo

An example run with the trace info (only the launcher spec):
https://github.com/puppeteer/puppeteer/actions/runs/4145033164/jobs/7168864438
https://github.com/puppeteer/puppeteer/actions/runs/4145033164/jobs/7168864939

Thanks for the analysis. Still it seems Firefox on mac is about 3x slower than Chrome on Mac. Are there easy some optimisations possible (maybe via some command line flags)? If not, then let's close it.

Unfortunately, we don't have anything like this at the moment :(
I think the time should go down generally for Firefox, when more tests start passing.

@lutien are you going to spend some time on getting the tests to pass? I thought you would not spend much time on the CDP support now. So perhaps it makes sense to skip tests that won't get fixed? Also, in the launcher spec example most or all tests are passing but Firefox test time seems to 2.5m vs 30sec for Chrome.

Please note that this is most likely related to slow startup times of Firefox and maybe some shutdown delays. As we talked on Matrix a while ago it would be great to see a CI job on MacOS (launcher spec tests should be enough) that have full trace logs ('remote.log.level=Trace`) enabled. This potentially could indicate what's slow - maybe it's related to the workers GitHub is using.

@whimboo there are two runs with the trace info linked above:
https://github.com/puppeteer/puppeteer/actions/runs/4145033164/jobs/7168864438
https://github.com/puppeteer/puppeteer/actions/runs/4145033164/jobs/7168864939

So generally it takes ~3s for Firefox to startup on MacOS. Given the amount of tests this adds quite a good amount of extra time.
For userDataDir option should restore state it means that Firefox runs this test in 9985ms while Chrome only takes 1426ms. That alone makes a difference of ~8.5s because the browser starts twice (~6s extra time for Firefox).
We are aware of slowness during startup and there is a bug filed for that (which I actually cannot find), but there is nothing easy to do right now to get this solved.
One thing that we could try is to create a specific Firefox profiler export for a test that launched Firefox once. We would have to measure startup and shutdown. The exported JSON file would have to be marked as an artifact so that we can upload to our profiler for analysis. If you could run that I'm happy to have a look at the results.


Migrate build from kbuild to kdist

Source and documentation: https://github.com/runtimeverification/pyk/tree/master/src/pyk/kdist
As a kdist target executes arbitrary Python code, it also allows for generation of artifacts other than kompiled definitions, e.g. the GLR parser.

The current kbuild implementation works fine for us.
Since we have other high priority tasks, we should delay addressing this issue to a future time when necessary.


Download page must use HTTPS for sigs and hashes

The download page must use HTTPS for sigs (.asc) and hashes (.sha512/.sha256).

Fixed, thanks for reporting this.
Not sure how we missed that.


pink-text css class created but tests is unsuccessful

Challenge Prioritize One Style Over Another has an issue.
User Agent is: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.110 Safari/537.36.
Please describe how to reproduce this issue, and include links to screenshots if possible.
My code:

Hello World!

Seems like the test is acting weird if the semicolon after the pink is missing. Not sure why this is happening though.

@redrexdas @BerkeleyTrue it appears the regex requires the semi-colon.
/\\.pink-text\\s*\\{\\s*color\\s*:\\s*pink\\s*;\\s*\\}/g

While it's not necessary in this case, is it something we want to enforce, or should we modify the regex?

@sludge256 Is correct, we should enforce the semicolons.
Happy Coding


Translate May 15 covid report into Greek

Happy weekend @nextstrain/greek,
The situation report from 2020-05-15 is ready to be translated to Greek. This week's report is short and focuses on mutations. It's targeted more at the public than some of our previous reports and is intended to answer some of the many questions we've been receiving on the subject. I hope it will be useful for Greek speakers.
Sadly, this will be our last weekly Situation Report for awhile although we'll issue periodic sit-reps as needed. Thank you, thank you, thank you for translating these reports. It's been an amazing collaborative effort to be involved in. I'll miss working with you on them.  I've personally derived a lot of global solidarity from our work together. Although we will all probably enjoy having more time on the weekend!
Branch to translate on: sit-rep_el
File to translate: https://github.com/nextstrain/translations/blob/sit-rep_el/narratives/translations_ncov_sit-rep_el_2020-05-15.md You can translate directly in this file, but please remember to push often as other people may be working on the translation!
Link to preview: https://nextstrain.org/community/narratives/nextstrain/translations@sit-rep_el/ncov/sit-rep/el/2020-05-15
Twitter post to translate:
“Our latest situation report focuses on viral mutations: why they occur, when they matter, and how they can be useful in tracking outbreaks. The report is now available in Greek.
Since repeated patterns of viral introduction and spread across the world have become clear, we'll be pausing weekly situation reports for a while. We will publish periodic situation reports as the data allows.
Thank you to @, @, @ for their rapid translation of this report and for their translations of previous reports."
Regards,
Cassia
Logistics
Please update the translations board as you work on the translation: https://github.com/orgs/nextstrain/projects/7
Please assign yourself to the translation if you’re working on it this week.
When a translation is ready for publication -- at minimum meaning two people have worked on it and both agree that it’s ready -- please create a pull request to merge to master and tag me and the issue in it!
Discussion board for your team is here: https://github.com/orgs/nextstrain/teams/greek
Standard rules for translation:
Note that the file to translate is in Markdown. In general, it is important not to change any formatting or links. (Although it is okay to update Wikipedia or CDC links to the appropriate language version.) Comments to help with translation are noted in comment symbols like <! -- Comment -->.
Please add your name under translators: with new line, tab, and dash, formatted just like authors are. Please also add a link to your professional page (can use Twitter) under translatorLinks:.
Note: You may know of other great resources available in Greek; please don’t add them to the situation report. We need to keep content the same across all translations, so we can be confident in the accuracy of what we publish. Thanks for understanding.
At minimum, I require two sets of eyes on every translation; one of those people should have a background in phylogenetics. Although if you have more, that’s fine too.
Feel free to use a live, online editor like hackmd (sign in with GitHub) to translate together.

Great report @cassiawag !
@codergr I have prepared a hackmd file with a line-by-line translation keeping the english text in there. Report is ready for review :)
https://hackmd.io/w0umeOncSFmIvVbitRDPWw?edit

Thanks @akifoss and @cassiawag excellent work !
"Η πιο πρόσφατη αναφορά κατάστασης επικεντρώνεται σε ιογενείς μεταλλάξεις: γιατί συμβαίνουν, πότε είναι σημαντικές και πώς μπορούν να είναι χρήσιμες για την παρακολούθηση επιδημιών. Η έκθεση είναι τώρα διαθέσιμη στα Ελληνικά.
Δεδομένου ότι τα επαναλαμβανόμενα μοτίβα της ιογενούς εισαγωγής και της εξάπλωσης σε όλο τον κόσμο έχουν καταστεί σαφή, θα θέσουμε σε παύση τις εβδομαδιαίες αναφορές κατάστασης για λίγο. Θα δημοσιεύουμε περιοδικές αναφορές κατάστασης, δεδομένων επιτρέποντος.
Ευχαριστούμε @akifoss και @codergr για τη γρήγορη μετάφραση αυτής της αναφοράς και για τις μεταφράσεις των προηγούμενων αναφορών. "

Published and tweeted: https://nextstrain.org/narratives/ncov/sit-rep/el/2020-05-15
Excellent work @akifoss & @codergr!


Rukshi patch dev

Update the Integrating Services tutorial.

This was sent by mistake. Hence, closing it.


Potential Code-Refactor and Bug Fixes

@HarshCasper thanks for your interest in arkade.
If you'd like to contribute, then please see the contributing guide for how to do so, you're welcome to raise an issue to propose changes. Do make sure you fill out the issue and PR templates in full.
https://github.com/alexellis/arkade/blob/master/.github/ISSUE_TEMPLATE.md
https://github.com/alexellis/arkade/blob/master/.github/PULL_REQUEST_TEMPLATE.md
Alex

/add label: invalid


vEth interfaces performance

HW: VMware 6.5 VMs (vmxnet3 NICs)
OS: CentOS 17.08
Kernel: 3.10.0-693.el7.x86_64
Contiv-VPP Build: latest (as of 7/3/2018)
I've been doing some installation and performance testing on CentOS using iperf3 containers. The initial goal was to see what the performance of intra-node containers are using VPP with both TAPv2 and AF-Packet (veth) interfaces.
contiv-vpp.yaml
iperf-server.yaml
iperf-client.yaml
When using TAPv2, I'm getting ~2.1 Gbps between containers on the same node.
However when using veth interfaces, I'm only seeing ~20 Mbps. These results are the same when setting TCPstackDisabled: True and TCPstackDisabled: False. I would have expected the AF_PACKET throughput to be significantly higher than TAP interfaces.
Also, I've noticed the CRI-shim has been commented out in a bunch of places in the code. A lot of the documentation still makes reference to it. Is this now deprecated? (if so, I can submit a PR to pull it out of the difference documentation areas)
Thanks.

Hi,
your results are actually expected, TAPv2 interfaces on VPP (virtio-based) are supposed to be much faster than AF_PACKET. Better performance was the reason for introducing them to VPP and making them default instead of veths in Contiv/VPP.
Yes, CRI shim, as well as TCP stack on VPP has been deprecated. Feel free to cleanup the doc, we would welcome your PRs.

Hi,
currently there is no high-speed alternative to deprecated LDPRELOAD. We are currently focusing on improving the stability of the plugin. Once that is done, we mat re-start work on high-speed alternatives.

@rastislavszabo would you please elaborate the reasons for LDPRELOAD was deprecated? thanks!


FederatedStore Double Cache Persistence bug

Describe the bug
If a Rest Service FederatedStore is started and a sub-FederatedStore is added. The sub store could be added with no persistent cache. This would cause the sub-FederatedStore to become unusable. due to being fetched from cache-built-used each time and retains no information.
If a Rest Service FederatedStore is started with No persistent cache, a sub-FederatedStore could be created with A persistent cache.

This may have all been fixed by the implementation of https://github.com/gchq/Gaffer/issues/2890

This has been fixed.

The Cache is no longer splatted by an AddGraph.
The inner federated store which built from cache repeatedly, has its info persisting in cache, because the Outter FederatedStore is keeping the light on for the Cache Service in JVM.

This was revealed by the Federation Recursion test after #2890

Closed by https://github.com/gchq/Gaffer/pull/2904


data dir owned by root

Hi we are back to testing the AMQ playbooks. When I run the newest version of the playbook I cannot start since the data directory is owned by root. We are running an NFS mounted dir like this:
 sudo mount -t nfs amqtest2.file.core.windows.net:/amqtest2/amqdata /opt/amq/amq-broker/data -o vers=4,minorversion=1,sec=sys

This is our playbook:
---
- name: Playbook for Red Hat AMQ Broker
  hosts: all
  vars:
    activemq_ha_enabled: true

  collections:
   - middleware_automation.redhat_csp_download
   - middleware_automation.amq
  roles:
   - activemq

The problem is that the data dir is owned by root, including the pid file
[azureroot@vm-amq-ansible-1 data]$ ll
total 3
-rw-r--r--. 1 root       root         7 Nov 17 08:17 artemis.pid
drwxr-xr-x. 2 root       root       128 Nov 17 08:18 bindings
drwxr-xr-x. 2 root       root       320 Nov 17 08:18 journal
drwxr-xr-x. 2 root       root        64 Nov 17 08:17 large-messages
drwxr-xr-x. 2 root       root        64 Nov 17 08:17 paging

This is the resulting problem for the service:
azureroot@vm-amq-ansible-1 amq-broker]$ sudo systemctl status amq-broker.service
● amq-broker.service - amq-broker Apache ActiveMQ Service
     Loaded: loaded (/etc/systemd/system/amq-broker.service; enabled; vendor preset: disabled)
     Active: activating (start) since Thu 2022-11-17 08:31:32 EST; 40s ago
    Process: 119510 ExecStart=/opt/amq/amq-broker/bin/artemis-service start (code=exited, status=0/SUCCESS)
      Tasks: 46 (limit: 21602)
     Memory: 171.7M
        CPU: 5.499s
     CGroup: /system.slice/amq-broker.service
             └─119516 /usr/lib/jvm/java-11-openjdk-<IP_ADDRESS>.8-2.el9_0.x86_64/bin/java -Xms512M -Xmx2G -XX:+PrintClassHistogram -XX:+UseG1GC -XX:+UseStringDeduplication -Dhawtio.disableProxy=true -Dhawtio.realm=activemq -Dhawtio.offline=true -Dhawtio.roleP>

Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied
Nov 17 08:31:55 vm-amq-ansible-1 systemd[1]: amq-broker.service: Can't convert PID files /opt/amq/amq-broker/data/artemis.pid O_PATH file descriptor to proper file descriptor: Permission denied

I believe it would be much better to place the mount directory outside of the amq program folder since it makes development and reinstlling AMQ much easier

We have solved it ourselved by mounting the nfs share at /opt/amq/amq-broker/data/shared outside of the data dir. It works via the variable  activemq_shared_storage: true and activemq_shared_storage_path


Provide the lib as a .net standard library

Hi,
The Nuget package should already contain a .Net Standard 1.3 library which can be used, but I note this was not listed in the readme under supported platforms so I've updated that to reflect this.
Thanks.


⚠ Bofrost.be has degraded performance

In f3ffdcd, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 2215 ms

Resolved: Bofrost.be performance has improved in 4f116f6 after 305 days, 7 hours, 8 minutes.


feat: support default value option of column  like mysql

Which issue does this PR close?
Closes #250
Rationale for this change
Support default value expr.
What changes are included in this PR?

When creating table, ceresdb can support to define normal expr which can reference other columns defined before it like mysql.
When insert into table, ceresdb will fill the missing columns which define default value expr.
This pr depends on #187.

Are there any user-facing changes?

Support normal default value expr when create and insert into table.

How does this change test
Add unit tests.

The main function is ready for review now.
But this feature depends on #187, Shall we wait it to merge first?
cc: @ShiKaiWi @jiacai2050 @waynexia

Sorry, I'm running a little behind,  I will finish this pr as soon as possible

The main function is ready for review now.

@ygf11 How about setting it ready for review now?

@ygf11 Thanks for your contribution, and it is a great job. I have left some tiny comments, and you can fix them when you are convenient.


SB, CB locations and deciding how many tiles we are going to use

Hello,
I want to ask about three things:
1- I couldn't see a part in XML files that defines from how many tiles FPGA consist of. At annotation part we are building CLBs with connecting FF and LUTs together in it, making connections with physical definitions for I/O, CLB etc. . So at annotation part there is no info given as from how many CLBs the FPGA is going to be consist of.
So I want to ask:
1.1 Do architect decide how many tiles there is going to be in FPGA architect at OpenFPGA XML architect file or VPR  xml architect file. If it is like this, did I miss the part it is defined?
1.2 Or with using XML files we are just generating netlist for unique parts of FPGA and at back-end tool with using this unique logic frames -just one time or repeatedly- create overall FPGA. So in this way we could say FPGA architecture finalized at back-end tool.
If the answer to this question is second option I want to ask my last question:
2- How locations of SB, CBx, CBy is known at fpga_top.v ? I thought SBs, CBs and their locations are automatically generated according to CLB locations. But I also couldn't find a part in architect files that defines number of CLBs there is going to be used and locations of each CLB? Is it automatically handled by OpenFPGA? If yes, how? Or it is also done at back-end process?
3- I understood K means number of LUT inputs, but what does N mean in architecture file names?
Thank you in advance.

@WazaAbdulkadir

Number of CLBs and other programmable blocks available in an FPGA fabric can be found in the packing statistics of vpr/openfpgashell log file.
It is the architect who should decide the dimension of the FPGA arrary, for instance, 4x4. You can force VPR to apply the fixed dimension through <fixed_layout> syntax. See the following examples (pay attention to the arch XML, openfpga script and config file):

https://github.com/lnis-uofu/OpenFPGA/blob/1f0f13ce0d1be583aee2733c61eb4d308e45e6b0/openfpga_flow/tasks/fpga_bitstream/generate_bitstream/configuration_chain/device_96x96/config/task.conf#L20-L24

OpenFPGA will automatically identify the number of unique tiles (based on your routing architecture) and build the netlists for only unique tiles. You may just run physical design on unique blocks and then assemble them in the top-level. If you want flatten netlists, you can disable the option --compress_routing.

SB/CBx/CBy modules are named after their coordinates. You can find details about naming at :https://openfpga.readthedocs.io/en/master/manual/fpga_verilog/fabric_netlist/#routing-blocks

-N indicates the number of LEs per CLB.


fix(contract): remove token delegation threshold and fix lock

Eliminated the token threshold mechanism from the River contract, including the removal of the MIN_TOKEN_THRESHOLD variable and the setTokenThreshold function. Updated related tests to reflect these changes, ensuring consistent functionality throughout the contract.
Replaced IntrospectionBase and LockBase with IntrospectionFacet and LockFacet for better modularization. Removed redundant lock methods and corrected minor typos in comments for improved code clarity and maintainability.
Update logic in _setDelegators to handle edge cases for delegate assignment more efficiently. Simplify River._delegate by reusing the current delegate storage.
Fix delegation lock and add tests.

@shuhuiluo How do you intend to deploy the changes to the River contracts, which are not setup as diamonds in base, mainnet given the token has already been created ? A short runbook of how these changes will make it to production and effects on current token distribution should be added to this PR.

@shuhuiluo How do you intend to deploy the changes to the River contracts, which are not setup as diamonds in base, mainnet given the token has already been created ? A short runbook of how these changes will make it to production and effects on current token distribution should be added to this PR.

The token contract will need to be redeployed on Base. @pfives will make a migration plan.


[FEATURE]Add data-prepper pipeline for AWS-ELB Integration

Is your feature request related to a problem?
Add a data-prepper pipeline for the aws-elb integration - this should include

README describing the setting up of the data-prepper-pipeline
Local docker compose sample for a file source pipeline
actual (s3) pipeline that may be used for the logs ingestion

Do you have any additional context?

https://github.com/opensearch-project/dashboards-observability/pull/772

Local docker compose sample for a file source pipeline

Data Prepper has a log_generator source.
Could the Docker compose use a log_generator instead of a file source? Or is the testing framework generating a file and thus this needs to be a file?


netsh hangs when run from PowerShell 7.1.3 in Windows Terminal, cannot repro with new tabs

Environment
Windows build number: 10.0.18363.1377
Windows Terminal version (if applicable): 1.7.572.0 Preview

PowerShell 7.1.3

C:\WINDOWS\system32\netsh.exe, product version 10.0.18362.1

Steps to reproduce
Run C:\WINDOWS\system32\netsh.exe
Expected behavior
Input should be possible at the following prompt
netsh>
Actual behavior
No input is possible, the netsh.exe process needs to be terminated.
This behavior cannot be reproduced:

in PowerShell 7.1.3 not running within Windows Terminal
in other / new Windows Terminal tabs running PowerShell 7.1.3
in Windows PowerShell 5.1 running within Windows Terminal
on another machine running Windows 10 build 19042.867

So there is something special about this particular instance of PowerShell 7.1.3 running in that particular Windows Terminal tab.
I can keep that tab open as long as needed in order to diagnose the issue.

Trying to get a call stack on the "hung" netsh.exe process, but this is all I see:

I was forced to restart the machine, so the blocked tab is gone. Sorry.
I have a dump of the hung netsh.exe process, if that can help.

I mean it might be helpful, but we're not really the owners of netsh.exe, so I'm not sure having me poke around in a dump of that executable is going to lead to anything particularly informative. Not really sure where their bugs live though...

Dump might help -- though I bet it just uses 'ulib' or whatever internal console reading library all those old tools use. I wonder what they have in common...

How do you want to get the dump?

I mean it might be helpful, but we're not really the owners of netsh.exe, so I'm not sure having me poke around in a dump of that executable is going to lead to anything particularly informative. Not really sure where their bugs live though...

I understand. But that's the second executable that hangs when run within Windows Terminal...

E-mail would be fine. Thanks!

That is- using the e-mail address on my GitHub profile!

Sent!

Yup. This netsh is just waiting in the CRT read function on stdin.
Next time this happens, can you grab a dump of the OpenConsole.exe servicing the tab that's not working? It'll shed more light on the situation.
(Please reopen/open anew if you do repro this again!)

Yup. This netsh is just waiting in the CRT read function on stdin.
Next time this happens, can you grab a dump of the OpenConsole.exe servicing the tab that's not working? It'll shed more light on the situation.
(Please reopen/open anew if you do repro this again!)

Sure will do!

I don't know if it's the case for this particular hung process. But in general, with the addition of ConDrv in Windows 8.1, some operations are handled in the kernel I/O manager and device stack that used to fail immediately in user mode with ERROR_INVALID_HANDLE. Whether the operations are supported or not, the I/O manager attempts to acquire the file lock for synchronous I/O, which will block if the file is already acquired for another operation on another thread, possibly in another process. This is vulnerable to scenarios that behave like a deadlock, depending on the interaction of two or more threads. A lot of code is defensive against deadlocks and indefinite blocking when working with FILE_TYPE_PIPE, because of numerous warnings about pipes in the documentation, but not FILE_TYPE_CHAR such as console handles.
For example, say that the same file object for "\Device\ConDrv\Input" is inherited by multiple processes, and one thread has the file acquired for a blocking read in line-input mode. Another thread in another process might hang in SetFilePointerEx (i.e. query/set the NT FilePositionInformation) for a seek on the file from the beginning or current position. (In legacy standard I/O mode, Python's startup code is currently vulnerable to this.) Prior to ConDrv, calling SetFilePointerEx on a console handle would fail immediately with ERROR_INVALID_HANDLE.  In Windows 8.1 and later, with the ConDrv device, once the file lock for synchronous I/O is acquired, it trivially succeeds since the I/O manager just updates the position information in the file object without even calling the driver.
I think that most I/O system calls on console files that were formerly caught in user mode and failed with ERROR_INVALID_HANDLE (check the old source tree in Windows 7) -- such as GetFileInformationByHandle (i.e. NtQueryVolumeInformationFile: FileFsVolumeInformation, NtQueryInformationFile : FileAllInformation) and GetFileTime (i.e. NtQueryInformationFile : FileBasicInformation) to name two more -- should be failed early in the I/O manager by checking whether the file device type is FILE_DEVICE_CONSOLE before attempting to acquire the synchronous file lock. It can fail with STATUS_INVALID_HANDLE to be compatible with the old implementation. Or use a more appropriate status such as STATUS_INVALID_INFO_CLASS (i.e. ERROR_INVALID_PARAMETER) or STATUS_INVALID_DEVICE_REQUEST (i.e. ERROR_INVALID_FUNCTION).


chore: Remove en-us from azure.microsoft.com links

Find/replace https://azure.microsoft.com/en-us/ -> https://azure.microsoft.com/ to match link locale guidelines

@nschonni : Thanks for your contribution! The author, @jasonwhowell, has been notified to review your proposed change.

@nschonni @Sara-Nicolini - This sort of change would be best sent through the private repo.

@tysonn I'm not an MS employee so I can't see the other repo. I've got no problem with you folks just redoing internally next time though

@nschonni - Sorry, I thought you were part of localization, this is a task they are supposed to perform periodically. We have some validation in the works to prevent it in the future.

@nschonni And also, thanks for the fixes!

Yup, looking forward to https://docs.microsoft.com/en-us/contribute/validation-reference/locale-specific-links being turned on in repos 😄
If it's easier, you can pull/redo the other few similar PRs I have open for similar changes

We're testing it on https://github.com/dotnet/docs/pulls already @nschonni!


Earley parser produces non-deterministic results even with explicit terminal weights

Perhaps related to #191
When using the earley parser with weighted terminals, the result is still non-deterministic in certain cases.
Example:
from lark import Lark

grammar = """
root: A | B

A.1: ("0".."9")+
B.2: ("0".."9")+

%import common.WS
%ignore WS
"""

parser = Lark(grammar, start="root", parser="earley")
print(parser.parse("10"))

It is my understanding that using weights was a workaround for this non-determinism (as indicated in #167), but this does not seem to be the case.
I am using the latest lark version available on pip (0.6.2) on Fedora Linux 27 x86_64.

Earley currently does not support weights on terminals. But it should work as you expect it to with weights on rules. Such as:
root: a | b

a.1: ("0".."9")+
b.2: ("0".."9")+

This is probably not the best behavior. Hopefully I'll get the chance to fix it soon.

I see. Perhaps you could help me work around this issue with a project I'm working on.
In the project I have terminals similar to the following for unquoted strings, decimal numbers, and hexadecimal numbers:
root: STR | DEC | HEX

STR: /\S/+
DEC: ("0".."9")+
HEX: ("0x" ("0".."9" | "A".."F" | "a".."f")+)

I'd like to be able to have things like "10" always parse as a decimal and "0xFF" as a hexadecimal, but the non-deterministic behavior is causing them to sometimes be interpreted as strings.
Is there an easy way I can refactor this to work around the unexpected behavior?

Of course. Just add intermediate rules.
root: STR | DEC | hex

hex.2: HEX

etc.
Hope this helps!

Thanks for the advice. It seems to work in some cases but not in others.
For example, with the grammar and text below, the hexadecimal literal seems to be interpreted as a string unless I remove the string terminal entirely, in which case it is interpreted as a hexadecimal.
from lark import Lark

grammar = """
root: "BEGIN" statement+ "END"
statement: assignment

assignment: IDENTIFIER "=" rvalue
?rvalue: decimal
      | hexadecimal
      | STRING

IDENTIFIER: ("A".."z" | "0".."9")+

// basic types
STRING: /\S/+
DECIMAL: ("0".."9")+
HEXADECIMAL: ("0x" ("0".."9" | "A".."F" | "a".."f")+)

// wrappers for ambiguous types above
decimal.2: DECIMAL
hexadecimal.2: HEXADECIMAL

%import common.WS
%ignore WS
"""

text = """
BEGIN
    MY_VAR = 0xDEADBEEF
END
"""

parser = Lark(grammar, start="root", parser="earley")
print(parser.parse(text))

I'm not sure if this is an issue with my grammar or with the parser. The lalr parser is able to produce the correct behavior every time, but it is unfortunately unsuitable for the project I'm currently working on.
Any assistance would be greatly appreciated.

Hmm yeah, this is an actual bug with the current implementation.
I'm currently working on a new earley implementation that should solve this problem. You can check out the branch earley_sppf and try it for yourself. It's still under work, but it's fairly stable. Let me know how it works out for you!

I was able to switch over to lalr for the time being. Thank you very much for your help.
If you feel that this issue falls under the scope of #191, please feel free to close it.

0.7 published


Error: Cannot find module '@stdlib/string/base/trim' using @stdlib/bench

Description
Encountered an error when trying to require("@stdlib/bench")
Related Issues
No response
Questions
No.
Demo
No response
Reproduction

Install @stdlib/bench in a new project folder
Create a script containing require("@stdlib/bench")
Run it

Expected Results
Module loads
Actual Results
Error message:
Error: Cannot find module '@stdlib/string/base/trim'
Require stack:
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>- /home/pje/eventful/bench/harness.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1026:15)
    at Function.Module._load (node:internal/modules/cjs/loader:871:27)
    at Module.require (node:internal/modules/cjs/loader:1098:19)
    at require (node:internal/modules/cjs/helpers:108:18)
    at Object.<anonymous> (/home/pje/eventful/bench/node_modules/@stdlib/bench/harness/lib/benchmark-class/comment.js:23:12)
    at Module._compile (node:internal/modules/cjs/loader:1196:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1250:10)
    at Module.load (node:internal/modules/cjs/loader:1074:32)
    at Function.Module._load (node:internal/modules/cjs/loader:909:12)
    at Module.require (node:internal/modules/cjs/loader:1098:19) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>    '/home/pje/eventful/bench/harness.js'
  ]
}

Version
@stdlib/bench version 0.2.0
Environments
Node.js
Checklist

[X] Read and understood the Code of Conduct.
[X] Searched for existing issues and pull requests.

Further investigation shows that this is due to a few modules referencing @stdlib/string/base/something that should be @stdlib/string/something (trim and replace, specifically).  The modules in question (in @stdlib/bench) are harness/lib/runner/encode_assertion.js and harness/lib/benchmark-class/comment.js.

@pjeby As a workaround, install @stdlib/bench-harness instead. @stdlib/bench is a top-level namespace which exposes as its main export the benchmark harness.

@pjeby Update: we just triggered a patch release for @stdlib/bench. Can you try installing that package again?
We also discovered a small issue in the published @stdlib/bench-harness package which we are working to address.

Looks like 0.2.1 fixes the issue.  Thanks!

@pjeby Of course! Thanks for taking the time to file an issue!


pls dont rush the API 1.0 but do think properly and optimize for performance

i'm looking forward to write the storage backend for gomysqlserver with rocksdb so hopefully the api 1.0 will set the foundation forever and optimized for performance considerations so we dont have keep changing.

Yes, 1.0 will be our commitment that the API has solidified.


Nice Project can you add counting based on Vehicle classification type like car, Bike, Truck, Bus??

Can you throw more light on what exactly you want to accomplish? Some detectors (yolo, ssd) can classify vehicles, others can't (bgsub, haarc). It's just a matter of changing a few lines of code to get the classification working but I need to know what you want to do/have in mind.

mean if you separate  count car, separte count truck and separate count bus when they cross the ROI

Okay. It's possible.

Okay. It's possible. You can return bounding boxes along with the classes of objects detected. While this is easy to do, I feel it'll complicate the application as a whole. Any specific reason why you want this?
I think a better approach would be to save images of each vehicle once counted to be processed by another program solely created for vehicle classification. Images may also be used for other purposes such as vehicle identification (ANPR).

@nicholaskajoh  it would be great if you could tell where exactly to return the claass of the object?

You can return the classes in detectors/yolo/yolo_detector.py like so:
classes = [] # step 1

for output in outputs:
    for detection in output:
        # ...
        if confidence > 0.5 and classes[class_id] in classes_of_interest:
            # ...
            classes.append(classes[class_id]) # step 2
# ...
return _bounding_boxes, classes # step 3

You can return the classes in detectors/yolo/yolo_detector.py like so:
_classes = [] # step 1

for output in outputs:
    for detection in output:
        # ...
        if confidence > 0.5 and classes[class_id] in classes_of_interest:
            # ...
            classes.append(classes[class_id]) # step 2
# ...
return _bounding_boxes, _classes # step 3


⚠ Bofrost.es has degraded performance

In 23f55c8, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1913 ms

Resolved: Bofrost.es performance has improved in 0a1bc7f.


what is up with all this resetting?

recurrence for the most part

fixed here: https://github.com/harthur-org/brain.js/commit/3755e1bce901a3cd79904e478f965ce71e09bb65


Add test for ISBN-10 ending in X

Spawned from https://github.com/citation-file-format/citation-file-format/pull/335
Potential test case: https://github.com/citation-file-format/citation-file-format/issues/323#issuecomment-904296215

PRs were merged, closing the issue


Updated list all pods with -o wide comment

Minor updates to the comment of:
// List all pods in plain-text output format and include additional information (such as node name).
$ kubectl get pods -o wide

/assign @zacharysarah

lgtm

/lgtm
/approve


Produce a Docker image for 0.5.0

Create a Docker image running a preferably lightweight Linux distribution for the latest SEGS 0.5.0 release. The Dockerfile should expose TCP 2106 and UDP 7002-7004 and utilize a volume for configuration, data, and log files. The image should be published to the Docker Hub at https://hub.docker.com/. Xalops from Discord has expressed some interest in this task.

An updated Docker image for SEGS has been created. Details on how to use it can be found here: https://hub.docker.com/r/ldilley/segstainer/


java.io.ByteArrayInputStream behaves differently between JS and JVM?

The following snippet:
val arr = new Array[Byte](5)
val bais = new java.io.ByteArrayInputStream("abcde".getBytes)
bais.read(arr, 0, 5)
println(bais.read(arr, 0, 0))

Prints -1 on the JDK11 and 0 on Scala.JS 0.6.31.
It seems the JVM implementation goes against the spec of InputStream#read:

 If len is zero, then no bytes are read and
     0 is returned; otherwise, there is an attempt to read at
     least one byte. If no byte is available because the stream is at end of
    file, the value -1 is returned; otherwise, at least one
   byte is read and stored into b.

Not sure who we should be following - the spec or the implementation - but it would be nice if it did the same thing on both platforms (Haven't tried other versions of the JDK)

If it clearly goes against the spec, it's worth checking if a bug report exists for OpenJDK, and if not, open one. I've successfully got things fixed upstream that way in the past.

Hah, this might be a specification bug in ByteArrayInputStream. The doc of read says:

Reads up to len bytes of data into an array of bytes from this input stream. If pos equals count, then -1 is returned to indicate end of file. Otherwise, the number k of bytes read is equal to the smaller of len and count-pos. If k is positive, then bytes buf[pos] through buf[pos+k-1] are copied into b[off] through b[off+k-1] in the manner performed by System.arraycopy. The value k is added into pos and k is returned.

So the behavior is in spec, but the spec seems contradictory with the base class.


Allow content to stream when window loses focus

This isn't really a bug, possibly more so a feature request?
When the chat content is being streamed to the UI and updateState is called, if you switch to another window/tab the content will stop streaming and only resume when the window regains focus. I assume the browser is suspending the script execution? Is there a way around this? ChatGPT handles this well...

I would also be keen on seeing how this is done because I have noticed this issue as well.

I believe I found a way to implement this, but I am not sure how well it aligns with this project's code standards. In the updateState function in Chat.tsx there is a usage of setTimeout that is designed to simulate the slower streaming to the frontend. setTimeout does not follow its specified delay if it is an inactive tab and rather appears to run every second instead if it is inactive. To solve this, we would need something like Web Workers instead that run of a different thread. Here is an easy solution I found that uses: https://github.com/chrisguttandin/worker-timers
Run
npm install worker-timers
Then import this into Chat.tsx
import { clearInterval, clearTimeout, setInterval, setTimeout } from 'worker-timers';
I think the alternative would be to create your own worker that handles the streaming, but I think this is a nice and quick solution.

This works well as a drop in replacement for setTimeout. @pamelafox - what do you think?

On a related note... It seems the backend finishes returning content from the LLM faster than the frontend can render it. Even removing the timeout (which makes it quick "jumpy") doesn't make much of a difference...


Error: No available storage method found.

Hey there!
First off, thank you for putting this together. I've been struggling a bit attempting to get this whole set up working myself - so thank you!
I don't think this is an issue with your code, but I was wondering if you are experiencing the same problem. The issue has to do with user log in. The only thing I've really changed thus far is the graph.cool URL to my own as you suggested. (thank you for introducing me to that awesome tool btw!)  So when I try to log in using the login form, the request fires off just fine and the response comes back containing a token as expected. But it seemed as though the app's state was not reflecting the "successful" login. So after digging around and logging some things, I found this error getting thrown from the localForage.js file at line 2259. Are you seeing this as well? Or is your log in working properly? I've found a few github issues on the localForage's page about something similar, but haven't found something I understand well enough to fix ... haha.
I'm using the latest Chrome on a Mac with Sierra 10.12.6.

Hey @Matterful, Thanks for your words :)
Which version of RAN are you using? You can see it from "package.json" if you don't change it. Or you can check meta tag of it. Also, can you write the error log that you get?

Sorry about that. I am using RAN v0.3.1, so the version from a couple days ago. The error itself is happening silently in the background, and doesn't show up from regular logging. I first found it as I was chasing around the value of the token to see why the state wasn't getting updated. I added a console log into the reduxStore.js file just below the token variable definition on line 16. It looks like so:
     const token = await Promise.resolve(persist.willGetAccessToken());        console.log(reduxStore.js, token: ${token});
As it came out through the console, this is what it looked like:
reduxStore.js, token: Error: No available storage method found.

Hey @Matterful , I'm still investigating this so I will write after found :)

Hey @Matterful , I fixed it and It's on the master branch now. Can you try it please?

Awesome! Thank you good sir. I will give it a shot when I get some time today and let you know how it goes :)

Hey @Sly777, I've been having the same bug.
Since you fixed it, I now see that the app state changes properly immediately after I create a new user, but if I log out, then I try to sign in again the state does not update (so I can't see the "LogOut" link in the menu for example). I can see that I get a valid token back in the response - it's just the app state doesn't update.
If more detailed information would help you just ask me whatever you need - I'm happy to provide it.

Hello again!
Thank you again for working on this. I can confirm what GreatScot said. The sign up seems to work perfectly now, and logs you in correctly :) . But when just trying to sign in, the state doesn't seem to update.

@Matterful @GreatScot  fixed there https://github.com/Sly777/ran/pull/100

Hey @Matterful @GreatScot , I merged the changes now. It should work.

Thank you good sir!

Thanks much - it's working now.


SAK-41489 Sitestats for Students: Display the number of visits and th…

…e time spent on the site

Tests are broken.

Minor tweaks and fix tests, it looks good and sounds promising, thanks @josecebe, awesome work!!

Awesome @josecebe !!!!


Loader Doest go away

Simple Bug
Steps To reproduce it :
1) Go to Any Course Page

2) Click on Buy Now:

3) Cancel Payment:

4) You will see the Loading animation and doesn't go away:

Hey, this repo is only for app.100xdevs.com but the issue you are referring is for external service


Add steam.cdn.on.net to steam.txt

What CDN does this PR relate to
steam
Does this require running via sniproxy
no
Capture method
Reverse lookup of the IP Steam is downloading from when updating.


Smarter Banking Search

For something like:
val x = SRAM[Int](96,3,3,96)
Foreach(96 par 2){r =>
  Foreach(96) { c =>
    x(r,0,0,c)
    x(r,0,1,c)
    ...
    x(r,2,1,c)
    x(r,2,2,c)
  }
}

It is trying every possible power of 2 alpha combination and then the non power of 2 alphas, but visually inspecting this I think it should be relatively easy to jump straight to something like alpha=<9,3,1,0>.
Also there is no point in having AlphasX or Alphas2 contain two numbers if they are equal mod N

I thought about this a bit and I have a solution that works much faster for at least some of our banking-intensive apps.  Here is the new Alphas in exhaustive banking.
The idea is searching all powers of 2 first, then brute forcing all numbers next takes forever to hit on a valid scheme.  I think in the common case, the alphas will be either a power of 2, some factor of N, or a number that is a product of a subset of the dimensions (ex a 10x3x3 matrix has banks 0-8 on page 0, 9-17 on page 1, etc).  I added the AlphasLikely set to try searching through these first.
  /**
    * Generates an iterator over all vectors of given rank, with values up to N
    * Prioritizes vectors which are entirely powers of 2 first
    * Next prioritizes vectors composed of either
    *    1) Number that evenly divides N
    *    2) Number that is the result of a product of some combination of stagedDims (Not sure if this increases the likelihood of valid schemes compared to option 1 only)
    *    3) Number that is also power of 2
    *    (consider something like 96x3x3x96 sram N = 36, want to try things like alpha = 18,1,3,9)
    * Note that the total size is O(N**rank)
    */
  def Alphas(rank: Int, N: Int, stagedDims: Seq[Int]): Iterator[Seq[Int]] = {
    // Prime factors of number, for shortcircuiting the brute force alphas
    def factorize(number: Int, list: List[Int] = List()): List[Int] = {
      for(n <- 2 to number if (number % n == 0)) {
        return factorize(number / n, list :+ n)
      }
      list
    }

    val a2 = (0 to 2*N).filter(x => isPow2(x) || x == 1 || x == 0)
    val alikely = (
                Seq(0,1) ++
                Seq.tabulate(factorize(N).length){i => factorize(N).combinations(i+1).toList}.flatten.map(_.product) ++
                Seq.tabulate(stagedDims.length){i => stagedDims.combinations(i+1).toList}.flatten.map(_.product).filter(_ <= N) ++
                (0 to 2*N).filter(isPow2(_))
               )
    def Alphas2(dim: Int, prev: Seq[Int]): Iterator[Seq[Int]] = {
      if (dim < rank) {
        a2.iterator.flatMap{aD => Alphas2(dim+1, prev :+ aD) }
      }
      else a2.iterator.map{aR => prev :+ aR }
    }
    def AlphasLikely(dim: Int, prev: Seq[Int]): Iterator[Seq[Int]] = {
      if (dim < rank) {
        alikely.iterator.flatMap{aD => AlphasLikely(dim+1, prev :+ aD) }
      }
      else alikely.iterator.map{aR => prev :+ aR}
    }
    def AlphasX(dim: Int, prev: Seq[Int]): Iterator[Seq[Int]] = {
      if (dim < rank) {
        (0 to 2*N).iterator.flatMap{aD => AlphasX(dim+1, prev :+ aD) }
      }
      else (0 to 2*N).iterator.map{aR => prev :+ aR }.filterNot(_.forall(x => isPow2(x) || x == 1))
    }
    Alphas2(1, Nil).filterNot(_.forall(_ == 0)) ++ AlphasLikely(1, Nil).filterNot{x => x.forall(_ == 0) || x.forall(isPow2(_))} ++ AlphasX(1, Nil).filterNot(_.forall(_ == 0))
  }

A driving case was a 96x3x3x96 memory, where we access dims 0 and 3 with par 2 and all 9 elements along dims 1 and 2 in parallel.  The human solution would be to set alpha=18,1,3,9.  These alphas would eventually hit this, but they get to alpha=1,2,6,18 first which is also technically correct but a little more confusing.  I think playing around with the ordering of alikely and how the alphas get assembled should change this but I don't think it matters

Added uniqueModN on Seq implicits to get rid of alpha value collisions mod N

Good idea - I was meaning to get around to this improvement!


Nukie berf

Buff L6 firerate but require wielding for decent accuracy.
Nerf SMG accuracies so rifles more enticing.

Requires https://github.com/space-wizards/space-station-14/pull/16638 for the new component.
:cl:

tweak: L6 firerate is 6 again. It requires wielding to have decent accuracy.
tweak: SMG accuracy reduced to 3-tiles width at full screen away.

any berfers

omg me

fix tests


Esbuild minify options

Clear and concise description of the problem
When use esbuild to minify its mangle variable names. Some use case its break the running script for example minify an angularjs app.
Suggested solution
Just like terserOptions we can introduce a configuration so we can define what we want to minify.
esbuildOptions.minify = {
  /** Documentation: https://esbuild.github.io/api/#minify */
  minifyWhitespace?: boolean;
  /** Documentation: https://esbuild.github.io/api/#minify */
  minifyIdentifiers?: boolean;
  /** Documentation: https://esbuild.github.io/api/#minify */
  minifySyntax?: boolean;
}

Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

Very similar to #6065, but I think being able to have one less option is a plus. And would be fixed by #6178, so I'll close in favour of that instead.


Feature request: include parser feature in lambda layer

Use case
I want to use the new parser feature in my lambdas but need to ship it with my code as it is not available in the lambda layer currently.
Solution/User Experience
We simply need to include the feature when building / publishing the lambda layers.
Alternative solutions
Keep it out of the layer but update documentation:

It may be that this feature is not used frequently, in which case it makes sense to leave it out of the layers to minimize the layer size. In this case though, it should be noted in the docs that the parser feature is not included in the layer like the other features are.

Ship multiple layers:

If layer size is of concern, there may be a case for shipping a layer that includes the SDKs and the logger / metrics / tracer features, and then shipping individual layers for each of the other features that are used less frequently.

Acknowledgment

[X] This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
[ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers
Please react with 👍 and your use case to help us understand customer demand.

Note that this was also discussed in the Discord community.

Hi @scratchclaggy, thank you for opening the issue.
I'll assign the issue to you and will continue the discussion in the PR.
Thank you again!


xnxti side-effect clarification

from allen baum email:
If the CSR instruction that acccesses xnxti includes a write, the xstatus CSR is the one used for the read- modify-write portion of the operation,
while the xcause register’s exccode field, the xintstatus register’s xil field and the pending bit of the accepted interrupt
can also be updated with the new interrupt id, level, and clear respectively.
"clear" is and action; it should be a value (e.g. zero) - though only if it's an edge interrupt
(I suppose you could clear it even if it was a level, since my understanding is that it would have no effect and that simplifies the logic)
I'd like to see the phrase "as a side-effect" at the end of this.
But, the pseudocode implies that the only change to xstatus is the full set/clr/load of the data.
I'm just trying to make sure no other bits are updated. The following text does say that only horizontal traps are taken,
but without this, it would be possible to xret and clear the MPRV bit for example, then immediately re-interrupt.
This doesn't do that.

related to #100, #205?