Hrana disconnects with long lived streams

I'm facing the following errors:

https://github.com/tursodatabase/libsql/issues/985
https://github.com/tursodatabase/libsql-client-ts/issues/119

A sample log from my Dart application:
HranaException(400): Failed to run pipeline: "Received a baton that has already been used"
#0      HranaHttpClient._runPipeline (package:hrana/src/rpc_http_client.dart:84:7)
<asynchronous suspension>
simolus3/drift#1      HranaHttpClient.executeStatement (package:hrana/src/rpc_http_client.dart:161:22)
<asynchronous suspension>
simolus3/drift#2      Database._runSql (package:hrana/src/database.dart:78:12)
<asynchronous suspension>
simolus3/drift#3      Database.select (package:hrana/src/database.dart:92:22)
<asynchronous suspension>
simolus3/drift#4      _HranaDelegate.runSelect (package:drift_hrana/drift_hrana.dart:40:17)
<asynchronous suspension>
simolus3/drift#5      _BaseExecutor.runSelect (package:drift/src/runtime/executor/helpers/engines.dart:74:20)
<asynchronous suspension>
simolus3/drift#6      InsertStatement.insertReturningOrNull.<anonymous closure> (package:drift/src/runtime/query_builder/statements/insert.dart:174:22)
<asynchronous suspension>
simolus3/drift#7      InsertStatement.insertReturning (package:drift/src/runtime/query_builder/statements/insert.dart:152:9)

I see that there's a fairly short timeout of 10 seconds for stream ids on the server... So we should probably not be using that for the entire drift database haha.

This should be fixed in version 1.0.3 of drift_hrana. Please let me know if you still run into any issues with that version.

Thank you so much!


Mission dart:io import

Analyzer errors out on:
error: Undefined name 'Directory'. (undefined_identifier at [sass_builder] lib\sass_builder.dart:18)
Fix should be trivial, I'll do a PR.

Just noticed the same thing in 0.1.0 version. Thanks Daniel for fixing it.

Sorry, I was wrong. Still the same issue. To make it work again I had to revert lib to the previous version.


How to generate .npy files for different videos?

Is there a way to generate the .npy files for different videos other than the standard datasets existing?

i ask for the same problem

Hi, perhaps this helps: There's a public colab with sample code that shows how to run the RGB models on ucf-101 videos: https://colab.research.google.com/github/tensorflow/hub/blob/master/examples/colab/action_recognition_with_tf_hub.ipynb This is part of the tf-hub release of these models, links here: https://www.tensorflow.org/hub/modules/deepmind/i3d-kinetics-600/1 https://www.tensorflow.org/hub/modules/deepmind/i3d-kinetics-400/1 Joao
…
On Mon, Aug 6, 2018 at 12:28 PM, daixiangzi @.***> wrote: i ask for the same problem — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#26 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AO6qajIiQ-PaYqz-UBILXmuIQYII8_pLks5uOCh4gaJpZM4U-XiV .

Hi,
I just want to ask the same question. It helps a lot for me!
Thank you!
Chris


repo: normalize line termation across repo

This is a scrub of the entire repo to normalize end-of-line termination to
conform to the recently added .gitattributes file.  A mismatch of line
termination can cause difficult-to-resolve conflicts when rebasing or merging.
In this scrub of the repo, only two files actually required modification.
The process performed is described at:
https://help.github.com/articles/dealing-with-line-endings/#refreshing-a-repository-after-changing-line-endings
The .gitattributes file is required because some files (*.bat and *.sh) are
sensitive to line termination.  The .gitattributes file is stored in the repo,
and override settings which could be corrupted by a developer's local
git configuration.  The .gitattributes file was added in f077c81
This commit completes bcc9fec, which fixed a single file.

LGTM

The process described at the github is certainly correct.  For those with branches based on pulls of master in the incomplete state the process is to check out the affected branch (git status will show the list of affected as magically modified), disable the crlf in .gitattributes, checkout the affect files (s.t. they are no longer modified), checkout out the .gitattributes (undoing the disable), and then rebase to the the fully updated master.

Removing the .gitattributes would not fix the issue, unless the public history were rewritten such that .gitattributes never existed.  IMO this is a bug in git.
The problem originally being fixed was that LVL was depending upon the git "old system" of handling line termination via .gitconfig which is configured per-user.  Since the EOL selection is not in the repo, essential EOL state (on files that require it) is not guaranteed preserved.  Git's "new system" uses .gitattributes and is stored within the repo.  See http://adaptivepatchwork.com/2012/03/01/mind-the-end-of-your-line/

While removing .gitattributes doesn't work, you can comment out all
entries in .gitattributes and manually check-out the stubborn files and
then checkout .gitattributes with the same effect.
John
On Mon, Oct 30, 2017 at 10:59 AM, Mike Weiblen (LunarG) <
<EMAIL_ADDRESS>wrote:

Removing the .gitattributes would not fix the issue, unless the public
history were rewritten such that .gitattributes never existed. IMO this is
a bug in git.
The problem originally being fixed was that LVL was depending upon the git
"old system" of handling line termination via .gitconfig which is
configured per-user. Since the EOL selection is not in the repo, essential
EOL state (on files that require it) is not guaranteed preserved. Git's
"new system" uses .gitattributes and is stored within the repo. See
http://adaptivepatchwork.com/2012/03/01/mind-the-end-of-your-line/
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/KhronosGroup/Vulkan-LoaderAndValidationLayers/pull/2165#issuecomment-340511473,
or mute the thread
https://github.com/notifications/unsubscribe-auth/Ae91UhV6tprFbaYhPCnQw6b26yYee7upks5sxgBxgaJpZM4QJtnm
.


Behind Apache httpd Proxy

@sickill - First lots of thanks for Asciinema.
This is not an ideal case but this is what I have:

A CentOS Linux server running some sites - ports 80 and 443 are in use by regular httpd server.
I installed asciinema-server using docker-compose with following env vars in .env.production

URL_SCHEME=http
URL_HOST=localhost
URL_PORT=18880

docker-compose.yml - changed exposed port on host to 18880:

    ports:
      - "18880:80"

Defined reverse proxy at httpd level at host:

<VirtualHost *:80>
    ServerName www.markiv.us
    ServerAlias www.markiv.us
    DirectoryIndex index.php index.html
    SetOutputFilter SUBSTITUTE,DEFLATE
    ProxyPass / http://localhost:18880/
    ProxyPassReverse / http://localhost:18880/
    SetOutputFilter INFLATE;proxy-html;DEFLATE
    AddOutputFilterByType SUBSTITUTE text/html
    Substitute "s|http://localhost:18880/|https://www.markiv.us/|i"
    <Proxy *>
        Order deny,allow
        Allow from all
        Allow from localhost
    </Proxy>
RewriteEngine on
RewriteCond %{SERVER_NAME} =www.markiv.us
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:80>
   ServerName markiv.us
   RewriteEngine On
   RewriteCond %{HTTP_HOST} !^www\. [NC]
   RewriteRule ^(.*)$ http://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
RewriteCond %{SERVER_NAME} =markiv.us
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>

In above -
SetOutputFilter INFLATE;proxy-html;DEFLATE
AddOutputFilterByType SUBSTITUTE text/html
Substitute "s|http://localhost:18880/|https://www.markiv.us/|i"

I needed to add SetOutputFilter INFLATE;proxy-html;DEFLATE since it appeared that the response was compressed.
So - The site works fine as SSL termination is at the apache httpd level and not at the nginx.
Issue - 1
The issue is - when email comes after login - the url has http://localhost:18880.
Issue - 2
I tried using the following in .env.production
URL_SCHEME=http
URL_HOST=markiv.us
URL_PORT=80

The email that comes has above URL but asciinema player breaks it does not start.
Is there something that you can suggest for the config or env vars? I am close and thanks for your insight.

I guess it's too late but I'll try :)
This one seems correct to me:
URL_SCHEME=http
URL_HOST=markiv.us
URL_PORT=80

The port number here needs to be one to use for generating URLs, and given you're fronting it all with Apache on port 80 it should work like that.
What error do you get when the player doesn't start with these settings?

I guess it's too late but I'll try :)
This one seems correct to me:
URL_SCHEME=http
URL_HOST=markiv.us
URL_PORT=80

The port number here needs to be one to use for generating URLs, and given you're fronting it all with Apache on port 80 it should work like that.
What error do you get when the player doesn't start with these settings?

@sickill I am facing a similar issue. I'm not using apache but nginx instead. I have a Linux server that runs nginx as a reverse proxy, proxying multiple applications that each run on their own port (they may or may not run in docker containers). Because of that very use case doing rewrites (for the "/") is only possible once so this is a not a valid solution for my use case.
Is it possible to fix the source code instead to use relative paths instead of absolutes for all static/asset information?
This should be fixed at the application level not at a deployment level IMHO. This would fix the original issue with apache as well and no rewrites should be needed.
Big thank you in advance!

Like @ku1ik said this seems wrong:

URL_SCHEME=http
URL_HOST=localhost
URL_PORT=18880

The URL_PORT controls the port inside the container so
    ports:
      - "18880:80"

doesn't work because the way it's now configured should have this in the docker-compose.yml:
    ports:
      - "18880:18880"

Just don't change the port using the variables at all and instead use docker to do it for you:
    ports:
      - "18880:4000"

Also you should change the host as localhost works only on your local PC. When you'll try to connect from another machine it will break as the server doesn't know it's hostname which you use for connections - that's why it's there. If you want to test it on the local network just set it to your server's hostname in the network. That way you can connect to the machine similarly to this, depending on your local DNS suffix: myserver.local
URL_HOST=myserver.local

I just set asciinema like so (using traefik but it should work the same way with apache):
URL_HOST=asciinema.myserver.com
URL_SCHEME=https

    labels:
      - traefik.http.routers.asciinema.rule=Host(`asciinema.myserver.com`)
      - traefik.http.routers.asciinema.entrypoints=websecure # Port 443
      - traefik.http.routers.asciinema.tls.certresolver=lets-encrypt # Enable SSL between proxy and internet
      - traefik.http.routers.asciinema.service=asciinema
      - traefik.http.services.asciinema.loadBalancer.server.port=4000 # What port to connect to in the container. You'll have to expose it like I said above

AFAIK the server doesn't care what you set the variables to. It uses it for generating urls, etc. You can run it on the default port (like I do) and then just use docker + apache to set a different port and proxy it. Also your proxy config seems overly complicated. Just point the proxy to the server and enable SSL, nothing else should be needed.

@Ciberth

proxying multiple applications that each run on their own port ... Because of that very use case doing rewrites (for the "/") is only possible once so this is a not a valid solution for my use case.

Why don't you use a subdomain like asciinema.yourserver.com? It's easy to set up using nginx and it's more readable URL IMHO:
server {
    listen      80;
    listen      [::]:80;
    server_name asciinema.yourserver.com;

    # logging
    access_log  /var/log/nginx/access.log combined buffer=512k flush=1m;
    error_log   /var/log/nginx/error.log warn;

    # reverse proxy
    location / {
        proxy_pass            http://localhost:4000;
        proxy_set_header Host $host;
    }
}

It would be nice to have a subdir option as well but I've been using subdomains for almost everything and I find it more flexible and readable. For you it could be at least a temporary solution if you don't like the subdomain approach.

I've just released new version of the server: https://github.com/asciinema/asciinema-server/releases/tag/v20240515
One related change in this release is changing of the URL used for fetching recordings in the player, from absolute URL to a local (absolute) path. This should help with cases where the URL_HOST and URL_PORT are misconfigured.
However, it's essential to configure  URL_HOST properly for the whole installation work anyway (https://docs.asciinema.org/manual/server/self-hosting/configuration/#base-url), and as @jiriks74 suggested, when deploying together with other services it's best to keep them on separate subdomains. The server assumes it runs at the root of a domain, and it's likely going to stay this way (supporting nesting under arbitrary paths is a PITA to maintain, sorry).


Malformed Framework

Version: 1.18.0
OS: macOS

Looks like the structure of generated framework is invalid

Apparently this has only recently been enforced. The issue is with the onnxruntime package from the look of it, so we'll try to fix it there.
Do you see the issue when trying to upload an iOS app or a macOS app?

macOS app

Do we need to wait for next release to have this fixed?

You will need to wait until the 1.19 release. I think that will happen soon.


Spring Boot 3.1 compatibility

org.springframework.boot.autoconfigure.liquibase.LiquibaseAutoConfiguration changed in a backwards incompatible way in Spring Boot 3.1.
See: https://github.com/spring-projects/spring-boot/commit/d4980ea9931c6f48641dd0316d1b1737a50a4a7a
See: https://github.com/spring-projects/spring-boot/issues/34776

Thank you. This is invaluable. I'll have a look.

@candrews :  Version 1.4.0 has just been released .. with support for Spring Boot 3.1.  Thank you!


Installing modules throws errors

What is the proper way to install modules? I am getting this type of error:
Collecting micropython-ssd1306
  Using cached micropython-ssd1306-0.3.tar.gz (1.8 kB)

ERROR: micropython-ssd1306 from https://files.pythonhosted.org/packages/01/d0/0841d47772962c80af3ab178ef062ed2cd524cb99eb38463e669428402a8/micropython-ssd1306-0.3.tar.gz#sha256=d00cae1ff680e6ae504e258e700f002874b0d924b1a604253b6ea63144b58289 does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

I have been wondering this same thing. By the looks of it, you should use upip (docs found here), but running upip is throwing errors for me. I posted this issue here, but I'm wondering if it could be an issue with the plugin.

Does upip run on the microcontroller? If so, then it would likely need a network connection to connect to a repository. I have some Pi Pico W's on the way and this is one thing I plan to try.


DataGrid not computing proper widths for clr-dg-cell when width is applied through css in percentage

DataGrid not computing proper widths for clr-dg-cell when width is applied through css.
How to reproduce
https://stackblitz.com/edit/column-width-issues-1-0
Steps to reproduce the behavior:

Go to shared stackblitz
Click on Add action button.
New record is added.
Observe the width of the cells for the new row.
Width applied is very less and content is shrinked.
Width of header items is computed properly

Expected behavior
Width of row cells should be computed applied based on user provided values.
Versions
App

Angular: 7.2
Clarity: 1.0.4

Device:

Type: MacBook
Browser chrome
Version Latest

Additional notes
Header-renderer.ts : detectStrictWidth results in the width for cells which is not accurate when userwidth is in percentage.
This works with 0.13.x :
https://stackblitz.com/edit/clarity-light-theme-v013-column-widths

The recommended workaround is to use flex-grow settings on your columns.


Return error information for variants of SetupError

Inference and Loading variants of SetupError carry more information about the error that was returned, bubble it up so that we provide more information about the error that is being returned.

Converting to draft because of internal requirements. If the patch looks fine, when the mentioned internal requirements are lifted I'll mark it ready for review. Sorry for the noise!

Closing, following the rationale in https://github.com/bytecodealliance/wasmtime/pull/7157.


Add information tooltip about Servicing Profiles

On page - https://learn.microsoft.com/en-gb/DeployOffice/other/coming-soon-to-microsoft-365-admin-center
It is announced that Servicing Profiles will no longer support Office channel switching, it would be useful to have a backlink or additional information listed on this page to state this.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: d23ac821-9cb8-8855-598f-d1327df04fac
Version Independent ID: 78f2127a-0c6b-d236-a415-6215c582ce8b
Content: Change the Microsoft 365 Apps update channel for devices in your organization - Deploy Office
Content Source: DeployOffice/updates/change-update-channels.md
Service: o365-proplus-itpro
GitHub Login: @nicholasswhite
Microsoft Alias: nwhite

Hi @Lewis-Barry, we are planning to release a new feature into the Microsoft 365 Apps admin center which will allow admins to trigger channel changes on devices. We will update the article once the new feature is broadly available.


Int b 21373 validation submit move details task order with files

B-21373
Summary
This pr is to add disabling of the "Submit move details" and "Approve selected" buttons on the Move details page. It also adds and alert message /disabling for "Manage orders" /"Done buttons if there's no uploaded documents. A scroll to view is also added to focus on required fields when data is not entered into a required field.
Is there anything you would like reviewers to give additional scrutiny?
this article explains more about the approach used.
Verification Steps for the Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Have the Agility acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

[ ] Has the branch been pulled in and checked out?
[ ] Have the BL acceptance criteria been met for this change?
[ ] Was the CircleCI build successful?
[ ] Has the code been reviewed from a standards and best practices point of view?

Setup to Run the Code

Instructions for starting storybook
Instructions for starting the MilMove application
Instructions for running tests

How to test

Access the application
Login as a SC
Search for a Move (could be new or already created), navigate to the Move details page.
Scroll down to "Orders" section and click the "View and edit orders" button.
"Manage orders" and "Done" buttons should be disabled with a message "File upload is required" if there's no document uploaded.
Click on the "Save" button, should stay on the same page and receive "Please select one or more files"
Upload a document and click the Save button.
Should be navigated to the "Move details" page with "Submit move details" button enabled (if all required orders fields are entered).
Verify disabling of the Submit move details button by repeating step 4 and then deleting the uploaded document and click the "Cancel" button. When navigated back to the Move details page the Submit move details button should be disabled.
Login as a TOO
Repeat step 3 - 7
Should be navigated to the "Move details" page, select a shipment(Move management seems to be selected by default) and the "Approve selected" button should be enabled.
Verify disabling of the Approve selected button by repeating step 4 and then deleting the uploaded document and click the "Cancel" button.
Repeat step 12 and the "Approve selected button should be disabled.
Login as SC/TOO
Repeat step 3 and 4
Make sure some of the required fields are not filled out(e.g Date issued, Report by date, Department indicator, Orders number) and scroll out of focus any of the fields not entered.
Click on the "Save" button, should scroll to focus the first required field not entered.

Frontend

[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, Edge).
[ ] There are no new console errors in the browser devtools.
[ ] There are no new console errors in the test output.
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.
[ ] This change meets the standards for Section 508 compliance.

Backend

[ ] Code follows the guidelines for Logging.
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys.
[ ] Have been communicated to #g-database.
[ ] Secure migrations have been tested following the instructions in our docs.

Screenshots
https://github.com/user-attachments/assets/7e6a70d3-fc1c-4480-9b05-40fb131d1f06
https://github.com/user-attachments/assets/11e8831b-d502-4304-a2bc-0bbb7bbfd779
https://github.com/user-attachments/assets/b0d47815-b62e-4145-9ff9-8c52873543a0

So far, everything is looking great. I have noticed some ANDI flags on this page that I may need to open an issue for, but it's unrelated to this PR's changes:

Another issue unrelated to this PR's changes, these console errors are going to become annoying. Andi doesn't like it either. Separate issue inbound

So far, everything is looking great. I have noticed some ANDI flags on this page that I may need to open an issue for, but it's unrelated to this PR's changes:

We do have a handy feature that I have been throwing UI changes into. I think it would be appropriate to throw in a blurb in there about fixing these 508/ANDI errors and hopefully get them fixed in the near future.


Issue with displaying playback progress in a custom UI for ExoPlayer.

When using a custom UI slider as the progress bar, there are issues with smoothness and not work. Adding a Text display for the time does not update every second. Are there any good solutions to these problems?
https://user-images.githubusercontent.com/26084189/232313796-1c00db1a-23d5-47cc-859a-99c589eaf775.mp4

@atsushinee I'll check this. A bit occupied at the moment but will definitely have a look at this.

@atsushinee the implementation in main branch is quite different after implementing full screen exo player. You can check it out once and then let me know if any changes are required :)


Missing parentheses with tuple sections and operator sections

HLint will suggest turning flip (,) (+ 1) into (, + 1), which is invalid. It should instead turn it into (, (+ 1)).

Thanks for the report. I note that map (+ 1) . map (+ 2) gets the correct suggestion of map ((+ 1) . (+ 2)) so there must be something a bit funky with just that suggestion... I'll investigate.

Thanks for the report. I note that map (+ 1) . map (+ 2) gets the correct suggestion of map ((+ 1) . (+ 2)) so there must be something a bit funky with just that suggestion... I'll investigate.

Thanks for the report. I note that map (+ 1) . map (+ 2) gets the correct suggestion of map ((+ 1) . (+ 2)) so there must be something a bit funky with just that suggestion... I'll investigate.

Thanks for the report. I note that map (+ 1) . map (+ 2) gets the correct suggestion of map ((+ 1) . (+ 2)) so there must be something a bit funky with just that suggestion... I'll investigate.

Thanks! We have to deal with sections properly, and we were in remParen, but not fromParen or fromParen1. I made the latter two delegate to remParen, so hopefully fixed now.

Thanks! We have to deal with sections properly, and we were in remParen, but not fromParen or fromParen1. I made the latter two delegate to remParen, so hopefully fixed now.


(feat) task manager tabs & warning message

-I got the tabs working by just doing a GET request on every switch. Not optimal, but we can refactor later.


Do not transfer session data twice on write

https://github.com/Satont/grammy-typeorm-storage/blob/e4aaaef59fd0da66f96827d6e4a1d998c996a69e/src/index.ts#L23 looks like the session data is read and transferred entirely. All that is needed would be to find out if the specified key exists.
As it seems like EXISTS isn't implement in TypeORM yet (confer https://github.com/typeorm/typeorm/issues/2815) I suggest to specify only the key column to be returned from find.

True.
I can add some select, but i'm not sure does it will work with typeorm + mongodb. So i choiced that way to check does record is exists.

I'll write some additional tests for mongo i guess.

Just tested mongo, seems like it returns all keys instead of partical key. But it should work with SQL db's.
So, i created new release and published it on github. Also did PR for grammy website (https://github.com/grammyjs/website/pull/18)
If you wan't to discuss something, i'll keep that issue open.

If there is way to convert nodejs lib to deno i'll glad to make it, by the way. But i'm not using deno personaly.

Just tested mongo, seems like it returns all keys instead of partical key.

It's completely fine because it's a performance optimisation that we're talking about, and not an actual bug. I suggest to leave it as-is, unless someone discovers a problem. If a more efficient storage adapter is required for someone, then a native mongo adapter could be created.

If there is way to convert nodejs lib to deno

Given that all dependencies of a project have equivalents on both platforms, it's easier to develop for Deno, and create a Node.js build from it, e.g. using deno2node. However, TypeORM does not exist for Deno. It therefore makes more sense to develop a second, independent storage adapter that uses denodb or another ORM for Deno.

Given that all dependencies of a project have equivalents on both platforms, it's easier to develop for Deno, and create a Node.js build from it, e.g. using deno2node. However, TypeORM does not exist for Deno. It therefore makes more sense to develop a second, independent storage adapter that uses denodb or another ORM for Deno.

True, i totally forgot there is no TypeORM for deno, sorry.
So, i think everything just fine, so i close this issue. Feel free to create new one or keep talking in that, if you have some minds.

Sounds good! Thanks again for your work :)


Unable to configure the custom mapping using "ConstructUsing"

I'm not able to map the following custom mapping and this possible with automapper.  Any work around.
 config.NewConfig<List<MyResult>, MyResultDto>()
            .ConstructUsing(x =>
            {
              //do some custom stuff
               return new PcrResultDto()
               {

               }
            });

Use MapWith

Thanks, @goremykin, is this documented somewhere (or) are there any samples available that I can refer to?  I have a feeling that Mapster bit lack on documentation.

@Gopichandar, Wiki

Thanks alot @goremykin


import errors?

Description of the bug
Whenever I run the app and navigate to the url line 262 in manifest/manifest/api/app.py calls a metaseq resource. This isn't installed during setup.py which forces you to install metaseq from pypi which is a python2 library for genetic modeling.
To Reproduce
Steps to reproduce the behavior:
Build the manifest and run a module then go to the website at /
Expected behavior
App actually loads?
Error Logs/Screenshots
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/pkg_resources/init.py", line 349, in get_provider
module = sys.modules[moduleOrReq]
KeyError: 'metaseq'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 2525, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1822, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1820, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python3.9/dist-packages/flask/app.py", line 1796, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "//manifest/manifest/api/app.py", line 262, in index
fn = pkg_resources.resource_filename("metaseq", "service/index.html")
File "/usr/local/lib/python3.9/dist-packages/pkg_resources/init.py", line 1135, in resource_filename
return get_provider(package_or_requirement).get_resource_filename(
File "/usr/local/lib/python3.9/dist-packages/pkg_resources/init.py", line 351, in get_provider
import(moduleOrReq)
ModuleNotFoundError: No module named 'metaseq'
Environment (please complete the following information)

WSL2 on FROM nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04 image (docker)

Additional context
Add any other context about the problem here.

Is this still an issue?


pixi: 0.17.1 -> 0.19.1

Description of changes
https://github.com/prefix-dev/pixi/releases/tag/v0.18.0
https://github.com/prefix-dev/pixi/releases/tag/v0.19.0
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/3790


Dev/jpl/uwp file logging

GitHub Issue: #
PR Type
What kind of change does this PR introduce?

Bugfix
Feature

Description

Add file logging on UWP.
Fix log attachements not working on iOS.
Fix regression from previous PR.

PR Checklist
Please check if your PR fulfills the following requirements:

[ ] Interface members are XML documented
[ ] Documentation (XML or comments) has been added and/or existing documentation has been updated
[ ] Architecture documents have been updated
[x] Tested on all relevant platforms

Other information

Internal Issue (If applicable):

/azp run


Name the GNU/Linux executables to Ubuntu 20.04–22.04 executables

These executables are proven to not work on any other distros

There was an incompatible version of libharfbuzz on manjaro and that prevented the electron executable from launching. Yukon could still be used fine from a browser though and maybe a browser should be automatically launched when a problem is detected with ElectronJS.


better (?) string concatenation

For reasons that I don’t fully understand the code wouldn’t compile on my Xcode 8.3 because of an “Ambiguous reference to member ‘+’” error in line 52.
I wonder if this is a bug in the current swift release.
So I rewrote the code a bit to join an array instead to create a multiline string. Hope you don’t mind.
Great project! Do you plan to implement full HyperCard support? Do you need any help? :)

Thank you very much for your commits! I appreciate.
Sadly I don't plan to implement full HyperCard support, because

it would be long. For example it requires text editing, script parsing (complex and very tricky, though people from PXI already did it).

there is no more active development in HyperCard, I just made this soft to see the stacks.

the stacks would be changed, so they'd have to be saved a new format, that would be weird. We can't use the current format because we don't know it yet enough to make editions safely, to the data as well as to the resource forks.

I had only thought on some little things to improve the experience, like, for every stack, displaying a list of the resources, displaying the patterns, making a Cover-Flow like navigation, and, to have a bit of fun, letting the user move between cards with the old-fashion visual effects (to be selected from a menu).
But I'm not sure to make any update. At least in the next days I will correct the compilation chain (the release is not automated for the moment).

Back in the day I learned programming using HyperCard and I still have fond memories. For years I wanted to reinvent HyperCard for the internet age. Guess it might be fun to implement a bit more functionality.

For a reimplementation of the HyperCard editor, I'd recommend looking at StackSmith.

Yeah, there's the list of the HyperCard clones at hypercard.org.
You can make up your mind.


Continuing to get MCA errors on 06/06 Swift binaries.

The flow is that I can login through Bluepic and post an image just fine on initial use of the app. But any subsequent time after a login, I get image uploading errors like:
[ERR] [MobileClientAccess] Authorization header not found
[DBG] [MobileClientAccessCredentialsPlugin] authenticate :: failure

Did some research with pods and manual testing so it seems like a server issue

Contacted MCA lead about my issue. It seems the auth token is not being saved in the keychain properly or just gets reset on app launch or destruction. The MCA team is looking into it as a bug possibly.

After talking with the MCA team, they provided a fix for my issue. Images now upload every time even after app restart.

Ran some more tests on a physical device and it seems to be working.


GRL not handled correctly

Currently, the way we are using the GRL is by generating a trial assuming only the time profiles for time distribution, and then removing events not in good runs. This causes a bit of a headache when trying to inject a set number of signal events.
@mjlarson uses a different method in his package that I would like to implement here. The new process would be as follows:

generate background events
treat the good run list as a set of uniform distributions with different start and stop times, weighted relative to each other based on the event rates in each run.
multiply the the integrals of the good run list distributions and the background time profile distribution together to get probabilities for each run
for each event, randomly choose a run based on those probabilities to put the event into
then, for each run, use inverse transform sampling to randomly distribute the events in the run

This way, the injected signal is exactly what we expect, and this gives a coarse-grained accounting for the seasonal variation.

fixed in better grl


Require Julia 1.3

This PR drops support of Julia < 1.3 since other packages such as Tracker and NNlib only support Julia >= 1.3 which would require us to backport fixes if we want to support Julia < 1.3 properly.

Test fail due to the error fixed in https://github.com/TuringLang/Turing.jl/pull/1377.

There seems to be a problem with Julia 1.5 on Windows 32bit. There is no clear error message though, and tests seem to be fine with Julia 1.3 on Windows 32bit, so it might be a problem upstream or in some other package. Should we disable tests on Windows 32bit completely? Or just with Julia 1.5?

It’s helpful to test Win32 on at least one Julia release. It allows us to sport syntactical and numerical errors for some algorithms, e.g. HMC.

Wouldn't it be sufficient to test on Linux 32bit to spot numerical errors? It seems we can test both Julia versions there.

Yea, that's enough. I did not realise we are testing 32 bit on Linux.

All remaining test errors are due to https://github.com/TuringLang/Turing.jl/pull/1377.


Logout

Proposal to add [FEATURE] to to the site.
Feature Name
Logout
Functionality
Currently, I am prompted for a login but I do not see a way to log out.

Notes
I'm not sure where navigation belongs in the app

Now that I'm looking at it, I think clicking on Galaxy to log out and go to the home screen (presumably the galaxy) would be great!

Note: I see that Logout is possible from the /wish-summary page
Maybe we don't need logout from the kid's workflow?

I'll work on this. I'll add a sign-out link into the header since the Galaxy image has changed to another one.


kibana role not any user use plugin

Hi
Is there a way in Kibana permissions not anyone can use the plugin That has permission app?

Hi,
Unfortunately no.
Contribution is welcome 😉


CMake iOS Error

When I execute prepare_libs.sh  ，
The following error was reported
I followed the link（https://llm.mlc.ai/docs/deploy/ios.html） below step by step to execute it
I have tried adding
enable_language(C)
enable_language(CXX)
set(CMAKE_C_COMPILER gcc)
set(CMAKE_CXX_COMPILER g++)  to CMakeLists.txt
I found that the “ios” folder does not have  CMakeLists.txt
bo@BodeMacBook-Pro ios % ./prepare_libs.sh

sysroot=iphoneos
type=Release
'[' false = true ']'
rustup target add aarch64-apple-ios
info: component 'rust-std' for target 'aarch64-apple-ios' is up to date
mkdir -p build/
cd build/
cmake ../.. -DCMAKE_BUILD_TYPE=Release -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_SYSTEM_VERSION=14.0 -DCMAKE_OSX_SYSROOT=iphoneos -DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_OSX_DEPLOYMENT_TARGET=14.0 -DCMAKE_BUILD_WITH_INSTALL_NAME_DIR=ON -DCMAKE_SKIP_INSTALL_ALL_DEPENDENCY=ON -DCMAKE_INSTALL_PREFIX=. -DCMAKE_CXX_FLAGS=-O3 -DMLC_LLM_INSTALL_STATIC_LIB=ON -DUSE_METAL=ON
CMake Error at /usr/local/Cellar/cmake/3.28.1/share/cmake/Modules/Platform/iOS-Initialize.cmake:4 (message):
iphoneos is not an iOS SDK
Call Stack (most recent call first):
/usr/local/Cellar/cmake/3.28.1/share/cmake/Modules/CMakeSystemSpecificInitialize.cmake:34 (include)
CMakeLists.txt:2 (project)

CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!
**

How to fix this issue？
**

This command worked for me
sudo xcode-select -switch /Applications/Xcode.app/Contents/Developer

Sorry for the delayed response. @chenqi2013 would you like to try if @sherelynyap's suggestion works?


add in the ability to reuse cs if already installed

This pr adds in a check to see if cs (the Coursier native launcher) is already installed. Using the native launch is faster than using the jar based launcher. Plus a lot of scala users may have this installed already, so there is no need to install Coursier again. However, if it's not found, it just downloads it as it would have before.
I also updated the config in info to match the actual root patterns that are set in the default config.

After reading #200, if the install functionality goes away, this can obviously just be ignored and closed.

no harm in merging it meanwhile.


couldn't get the namespace resource, such as role, rolebinding

The namespace's resource role isn't stored in the namespaces folder, but the cluster-scoped-resources. As we can see below, the role resource did in there, but the omc couldn't get it.
jiazha-mac:~ jiazha$ cat must-gather/must-gather.local.1006115980782152064/017d95993583ba00c54c4198f6626e74584f9146dfbc0f6cc5c8d4c09eae1c98/cluster-scoped-resources/operators.coreos.com/operators/amq-broker-rhel8.cluster-tests.yaml |grep -i role -A 3
      kind: Role
      name: amq-broker-operator.v7.1-5HK4JCfzhBY5ZNGXrYHHtv0ig7R5ukbpb1fy92
      namespace: cluster-tests
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      name: amq-broker-operator.v7.1-5HK4JCfzhBY5ZNGXrYHHtv0ig7R5ukbpb1fy92
      namespace: cluster-tests
    - apiVersion: v1

jiazha-mac:~ jiazha$ omc use ~/must-gather/must-gather.local.1006115980782152064/
jiazha-mac:~ jiazha$ omc get role -n cluster-tests
No resources roles.rbac.authorization.k8s.io found in cluster-tests namespace.
jiazha-mac:~ jiazha$ omc get rolebinding -n cluster-tests
No resources rolebindings.rbac.authorization.k8s.io found in cluster-tests namespace.

Hi @gmeghnag , yes, thanks! But the oc client can get this kind of role and rolebinding resources. Or do we plan to support it in the future?
jiazha-mac:must-gather jiazha$ omc get operator -n cluster-tests amq-broker-rhel8.cluster-tests -o yaml
apiVersion: operators.coreos.com/v1
kind: Operator
metadata:
  creationTimestamp: "2024-01-18T09:48:07Z"
  generation: 1
  name: amq-broker-rhel8.cluster-tests
  resourceVersion: "247318"
  uid: 6eaf1f98-6910-450a-96c9-d7f885438c12
spec: {}
status:
  components:
    labelSelector:
      matchExpressions:
      - key: operators.coreos.com/amq-broker-rhel8.cluster-tests
        operator: Exists
    refs:
    - apiVersion: operators.coreos.com/v1alpha1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:57Z"
        lastUpdateTime: "2024-01-18T09:57:57Z"
        message: install strategy completed with no errors
        reason: InstallSucceeded
        status: "True"
        type: Succeeded
      kind: ClusterServiceVersion
      name: amq-broker-operator.v7.11.5-opr-1
      namespace: cluster-tests
    - apiVersion: v1
      kind: ConfigMap
      name: amq-broker-manager-config
      namespace: cluster-tests
    - apiVersion: apiextensions.k8s.io/v1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: no conflicts found
        reason: NoConflicts
        status: "True"
        type: NamesAccepted
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: the initial names have been accepted
        reason: InitialNamesAccepted
        status: "True"
        type: Established
      kind: CustomResourceDefinition
      name: activemqartemisaddresses.broker.amq.io
    - apiVersion: apiextensions.k8s.io/v1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: no conflicts found
        reason: NoConflicts
        status: "True"
        type: NamesAccepted
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: the initial names have been accepted
        reason: InitialNamesAccepted
        status: "True"
        type: Established
      kind: CustomResourceDefinition
      name: activemqartemises.broker.amq.io
    - apiVersion: apiextensions.k8s.io/v1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: no conflicts found
        reason: NoConflicts
        status: "True"
        type: NamesAccepted
      - lastTransitionTime: "2024-01-18T09:57:53Z"
        message: the initial names have been accepted
        reason: InitialNamesAccepted
        status: "True"
        type: Established
      kind: CustomResourceDefinition
      name: activemqartemisscaledowns.broker.amq.io
    - apiVersion: apiextensions.k8s.io/v1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:52Z"
        message: no conflicts found
        reason: NoConflicts
        status: "True"
        type: NamesAccepted
      - lastTransitionTime: "2024-01-18T09:57:52Z"
        message: the initial names have been accepted
        reason: InitialNamesAccepted
        status: "True"
        type: Established
      kind: CustomResourceDefinition
      name: activemqartemissecurities.broker.amq.io
    - apiVersion: apps/v1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:57Z"
        lastUpdateTime: "2024-01-18T09:57:57Z"
        message: Deployment has minimum availability.
        reason: MinimumReplicasAvailable
        status: "True"
        type: Available
      - lastTransitionTime: "2024-01-18T09:57:57Z"
        lastUpdateTime: "2024-01-18T09:58:28Z"
        message: ReplicaSet "amq-broker-controller-manager-794c48d7c7" has successfully
          progressed.
        reason: NewReplicaSetAvailable
        status: "True"
        type: Progressing
      kind: Deployment
      name: amq-broker-controller-manager
      namespace: cluster-tests
    - apiVersion: operators.coreos.com/v1alpha1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:54Z"
        lastUpdateTime: "2024-01-18T09:57:54Z"
        status: "True"
        type: Installed
      kind: InstallPlan
      name: install-4kfpv
      namespace: cluster-tests
    - apiVersion: operators.coreos.com/v2
      kind: OperatorCondition
      name: amq-broker-operator.v7.11.5-opr-1
      namespace: cluster-tests
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: Role
      name: amq-broker-operator.v7.1-5HK4JCfzhBY5ZNGXrYHHtv0ig7R5ukbpb1fy92
      namespace: cluster-tests
    - apiVersion: rbac.authorization.k8s.io/v1
      kind: RoleBinding
      name: amq-broker-operator.v7.1-5HK4JCfzhBY5ZNGXrYHHtv0ig7R5ukbpb1fy92
      namespace: cluster-tests
    - apiVersion: v1
      kind: ServiceAccount
      name: amq-broker-controller-manager
      namespace: cluster-tests
    - apiVersion: operators.coreos.com/v1alpha1
      conditions:
      - lastTransitionTime: "2024-01-18T09:57:39Z"
        message: all available catalogsources are healthy
        reason: AllCatalogSourcesHealthy
        status: "False"
        type: CatalogSourcesUnhealthy
      - status: "False"
        type: BundleUnpacking
      kind: Subscription
      name: amq-broker-rhel8-0c
      namespace: cluster-tests
jiazha-mac:must-gather jiazha$ omc get role -n cluster-tests
No resources roles.rbac.authorization.k8s.io found in cluster-tests namespace.

Hi @jianzhangbjz, omc already supports role resource, but maybe these resources are not collected in your must-gather, show you an example:
$ oc project openshift-config
$ oc adm inspect $(oc get role -o name)
Wrote inspect data to inspect.local.2446175589228838906.
$ omc use inspect.local.2446175589228838906
$ omc get role -A
NAMESPACE          NAME                                                              CREATED AT
openshift-config   role.rbac.authorization.k8s.io/cluster-cloud-controller-manager   2024-01-17T19:57:22
openshift-config   role.rbac.authorization.k8s.io/console-operator                   2024-01-17T20:06:18
openshift-config   role.rbac.authorization.k8s.io/coreos-pull-secret-reader          2024-01-17T20:02:02
openshift-config   role.rbac.authorization.k8s.io/ingress-operator                   2024-01-17T19:56:29
openshift-config   role.rbac.authorization.k8s.io/insights-operator                  2024-01-17T19:56:27
openshift-config   role.rbac.authorization.k8s.io/machine-api-controllers            2024-01-17T19:56:39


Make ceph/demo launch using existing data

I really love the ceph/demo docker image because it provides a quick and simple way to spin up ceph for a development environment.
The only problem is that it currently treats every launch as a new cluster. I want to be able to keep my data when the docker container is terminated and then restart it at a later stage.
This is the command I am using to run the container:
sudo docker run --net=host -v /work/data/ceph/config:/etc/ceph -v /work/data/ceph/data:/var/lib/ceph -e MON_IP=<IP_ADDRESS> -e CEPH_NETWORK=<IP_ADDRESS>/24 ceph/demo

Unfortunately, if I terminate the container and then launch it again, I get this error:
2015-07-19 06:33:20.903601 7fd39a6a9900 -1  ** ERROR: error creating empty object store in /var/lib/ceph/osd/ceph-0: (22) Invalid argument

I believe this is because the entrypoint.sh file treats each launch as a new cluster and will go and create the OSDs and MDS from scratch.
If would be awesome if the script has some checks in place so that it's able to "restore" if data exists rather than create a brand new cluster.

Hey @F21,
This shouldn't be that difficult. I'll see if I can find some time this week to work on that.


Task 90192: Carry out any necessary code refactoring

Worked on the following:

Bug 91938: Signs out when click on Close (X)
Task 90192: Carry out any necessary code refactoring
Removed warnings (mainly around useEffect)
Removed unnecessary imports
Removed unused variables

Will need testing all the screens, paying particular attention to switching tabs.
To test @EdiWeeks, please run (in a bash Window):
rm -rf node_modules
npm install
npm start

Tested @EdiWeeks

Tested, please merge @BenGWeeks


when i use consul serviceDiscovery  UpstreamHost not working

Expected Behavior / New Feature
When I use DownstreamHostAndPorts directly it works,The following two links point to different services.
article services: http://gateway.afuyy.com/test
live service: http://test.gateway.afuyy.com/test

when i use consul serviceDiscovery  UpstreamHost not working,Both of the following links point to the same service.
http://consulgateway.afuyy.com/test
http://test.consulgateway.afuyy.com/test

Specifications

Version:
ocelot version: 12.0.1
.netcore version: 2.0
Platform:
Subsystem:

Where is your consul configuration on ocelot.json ?

example:
{
  "ReRoutes": [
    {
      "DownstreamPathTemplate": "/api/values",
      "DownstreamScheme": "http",
      "UpstreamPathTemplate": "/getvalues",
      "UpstreamHttpMethod": [ "Get" ],
      "FileCacheOptions": { "TtlSeconds": 10 },
      "AuthenticationOptions": {
        "AuthenticationProviderKey": "AuthScheme",
        "AllowedScopes": []
      },
      "ServiceName": "valueapi",
      "LoadBalancerOptions": {
        "Type": "LeastConnection"
      }
    },
    {
      "DownstreamPathTemplate": "/api/auth",
      "DownstreamScheme": "http",
      "ServiceName": "auth",
      "UpstreamPathTemplate": "/api/auth",
      "UpstreamHttpMethod": [ "Get" ]
    }
  ],
  "GlobalConfiguration": {
    "RequestIdKey": null,
    "ServiceDiscoveryProvider": {
      "Host": "localhost",
      "Port": 8500,
      "Type": "Consul"
    },
    "RateLimitOptions": {
      "ClientIdHeader": "ClientId",
      "QuotaExceededMessage": "",
      "RateLimitCounterPrefix": "",
      "DisableRateLimitHeaders": false,
      "HttpStatusCode": 428
    },
    "DownstreamScheme": "http"
  }
}

on ocelot.global.json

Facing similar issue.

Same, so I came.

Use the same upstream pathtemplate
That's the problem.

Hi @wtf-boy !
Yes, first scenario is correct, the second is invalid.

usage of the same "UpstreamPathTemplate": "/test", is wrong but validator didn't raised the flag because of different "UpstreamHost" properties.
usage of the same "DownstreamPathTemplate": "/" could be wrong if you've registered live and article services in Consul config in a wrong way

I guess both article, live services point to one host & port pair in Consul config.
Please, double check your Consul config!

@wtf-boy
Also, I guess maybe Upstream Host feature doesn't work at all for Service discovery scenario. Then a normal matching rules select the first route.
But it should be checked... Yeap, #216 is quite simple.
It is hard to guess, and advice something, you have to provide the logs!
Now I see a misconfiguration issue.
Please, upgrade to latest version 22.0 and let us know testing results once again.

This issue will be fixed as part of my PR #1944

@sliekens  Steven,
I can't get it...
How is this issue related to #1496 in #1944 ?!..
Does Load Balancer key influence to finding appropriate service instance in Consul?
@ggnaegi  FYI

@raman-m  Yes, it's all related to route key collisions. The route key does not currently include UpstreamHost, so the two routes in the this issue will have the same route key.
Given the current route key format is UpstreamPathTemplate|UpstreamHttpMethod|DownstreamHostAndPorts,
then both routes will have the route key: /test|Get,Post| (last part empty)

My PR will fix this by extending the route key format to make them unique once again.

Does Load Balancer key uniqueness (bad now) influence to finding appropriate service instance in Consul?

@raman-m no, the route key is not used when reaching out to Consul. However, the route key is used as a cache key, to prevent repeated calls to Consul for the same route. So, repeated calls will only use cached routes, and the problem occurs when route keys from different routes collide with each other, the wrong cached route will be used.

Assigned to @sliekens  :wink:
Please take care of the user scenarios of this issue in unit & acceptance tests!
You can use Theory xUnit attribute.


Simple task.h question

can I call task.stop() from within the task? Do I need to pass handle to task in order to do this? Do I even need to do this or can I just return from the task?

Yes, you can. I would say you even can call vTaskDelete(NULL) from inside that task, but this can cause memory leak, so myTask->stop() is enough.

thanks. so I need to pass the handle to the task?

No, all its required is handled by Task class. Here m_handleTask is filed with data:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/Task.cpp#L75
so if you start task before you try to stop it then m_handleTask wont be nullptr and such this line will be executed:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/Task.cpp#L90
So, all you have to do somhere in your code is:

create task (Task* myTask = Task())
start task (myTask->start())
stop task when its no longer needed (myTask->stop())
again start task if required (myTask->start())

yes, get that. What I am not clear on is whether or not the task has a handle to itself?
What I'm doing is starting the task, task executes code and when it has completed its routine, deletes itself - the code that started the task doesn't care about the task after task->start().

myTask has handle to itself, and like i said its this line:
https://github.com/nkolban/esp32-snippets/blob/master/cpp_utils/Task.cpp#L75
to delete task you need to execute myTask->stop() somewhere or myTask will stay in loop

got it. thanks


ES flipped coordinates

The coordinates for es/25829, es/25830, es/25831 and es/32628 are in the wrong order.
The mappings look good:
"conform": {
   "lon": "lon",
   "lat": "lat",
   "number": "number",
   "street": "street",
   "postcode": "postcode",
   "format": "csv",
   "srs": "EPSG:25829"
}

But image shows:

I also checked the corresponding script to generate the cache files like, https://data.openaddresses.io/cache/es-32628.zip but mappings seams also correct:
https://github.com/openaddresses/openaddresses/blob/44795aec6b236ee6f7c09e2f609f63770ae9dd2d/scripts/es/gml_to_csv.py#L75
Maybe you guys @sbma44, @mcliment know something?

Sorry @StephanGeorg but no, I just did some fixes to the script but did not change the behaviour (AFAIK).

I checked the cached sources:

https://data.openaddresses.io/cache/es-25829.zip
https://data.openaddresses.io/cache/es-25830.zip
https://data.openaddresses.io/cache/es-25831.zip
https://data.openaddresses.io/cache/es-32628.zip

All have correct data:
lon,lat,number,street,postcode,admin
243655.84,4289398.03,1,CL ANGEL,3720,
243665.04,4289413.23,2,CL ANGEL,3720,
243655.24,4289409.63,3,CL ANGEL,3720,
243662.84,4289422.83,4,CL ANGEL,3720,
243653.94,4289427.23,6,CL ANGEL,3720,
243218.64,4289191.22,4,CL BILBAO,3720,
243232.24,4289177.52,6,CL BILBAO,3720,
243241.63,4289161.22,8,CL BILBAO,3720,

but in the final GeoJSON file the coordinates are in the wrong order:
{"type":"Feature","properties":{"id":"","unit":"","number":"1","street":"CL ANGEL","city":"","district":"","region":"","postcode":"3720","hash":"154be3a6b8e6908e"},"geometry":{"type":"Point","coordinates":[38.7160828,0.0516575]}}
{"type":"Feature","properties":{"id":"","unit":"","number":"2","street":"CL ANGEL","city":"","district":"","region":"","postcode":"3720","hash":"85e4e44f68b77e9f"},"geometry":{"type":"Point","coordinates":[38.7162222,0.0517576]}}

I noticed the same problem with this dataset: "de_sn_statewide-addresses-state.geojson.gz"


Adjust subgraph deployment assignment cost dynamically

Right now we're costing every subgraph identically, however, some (e.g. ENS) are much more resource intensive than others (e.g. Decentraland). As more subgraphs get added to a node, this might lead to scaling problems.
This metric could be based on factors like:

number of entities created / modified
number of events processed
amount of time required for past handlers

See #940 for one way we could use to measure subgraph cost.

We've completely changed our strategy for assigning subgraphs, and this ticket doesn't make much sense any more.


Allow customization of variables hash tables

fixes #765

This change is

Coverage increased (+0.02%) to 46.589% when pulling 8837cf93e202b330b5bb99faa80100f928b23a56 on aledbf:variable-hash-size into f00823fd8331ee767b01ac77eeb35196a3cefff9 on kubernetes:master.


Add new frontend

This PR implements a new ReScript/BuckleScript frontend for benchmark visualisation. Apologies in advance for a massive diff.
The reason I decided to implement a new frontend are:

I am more familiar with the BuckleScript stack which contributes to my productivity.
Having types on the front-end is very nice and leads to a better development experience. This should help us as the project grows.
I'm hoping to reuse the visual building blocks defined for benchmarks for other projects such as the Irmin daemon.

Changes

Added a new frontend-new project alongside the existing frontend. I suggest to keep both while the new frontend is being tested and improved.
Implemented GraphQL queries for the new benchmarks database table. This resolves an issue that exists in the current frontend implementation where the metrics are hardcoded.
Added two UI helper modules:

Sx (Styling Extensions) - for utility-based CSS definitions. This is inspired by TailwindCSS.
Components - a small component library built on top of Sx.
My goal is to extract these into a separate library in the future.

The new frontend makes several changes to the way the benchmark graphs are laid out:

Each benchmark test gets its own section and not just a single graph.
Multiple benchmark metrics are rendered as separate graphs. This is mostly motivated by readability and the fact that the time series have different units. It should be possible to overlay multiple graphs if there're requirements for that.
The x-axis presents commits over time. Each commit label is a link that goes to the respective commit.
The graphs limit the rendered data to 20 commits, this is easy to change but I'll add support for a date-picker in the future.

Currently the frontend is configured to make requests directly to http://autumn.ocamllabs.io:8080/v1/graphql. To start the new frontend locally install the dependencies with yarn and then yarn start. Note that the GraphQL secret has to be set in index.res first.
Here's how the new frontend looks:

That looks really great! Just a question regarding the "20 commits". I think it would be useful to show the associated tags too (e.g. master or pr/234) and/or to add the ability to compare with a specific tag or a specific commit. Not totally sure how to display this, but the use-case is: I want to know how much the current PR improve things vs. the current master, and using only commits there is often not very practical.

That looks really great! Just a question regarding the "20 commits". I think it would be useful to show the associated tags too (e.g. master or pr/234) and/or to add the ability to compare with a specific tag or a specific commit. Not totally sure how to display this, but the use-case is: I want to know how much the current PR improve things vs. the current master, and using only commits there is often not very practical.

This looks great to me too! Will we have different page views for different PRs (as we have in the old front-end)?

This looks great to me too! Will we have different page views for different PRs (as we have in the old front-end)?

Just a question regarding the "20 commits". I think it would be useful to show the associated tags too (e.g. master or pr/234) and/or to add the ability to compare with a specific tag or a specific commit. Not totally sure how to display this, but the use-case is: I want to know how much the current PR improve things vs. the current master (or vs. an other PR), and using only commits there is often not very practical.

I definitely agree with this. The current view is centred around individual commits, but those should be grouped as PRs and be compared to master by default, as you explained.
I want to make these changes incrementally, starting with feature parity with the existing UI. I'll likely make some quick mock ups of the UI before implementing them to validate the use cases.

Just a question regarding the "20 commits". I think it would be useful to show the associated tags too (e.g. master or pr/234) and/or to add the ability to compare with a specific tag or a specific commit. Not totally sure how to display this, but the use-case is: I want to know how much the current PR improve things vs. the current master (or vs. an other PR), and using only commits there is often not very practical.

I definitely agree with this. The current view is centred around individual commits, but those should be grouped as PRs and be compared to master by default, as you explained.
I want to make these changes incrementally, starting with feature parity with the existing UI. I'll likely make some quick mock ups of the UI before implementing them to validate the use cases.

Will we have different page views for different PRs (as we have in the old front-end)?

Yes! Do you think that each PR view should only render commits for that particular PR? It seems like the current version renders all parent commits in addition to that?

Will we have different page views for different PRs (as we have in the old front-end)?

Yes! Do you think that each PR view should only render commits for that particular PR? It seems like the current version renders all parent commits in addition to that?

Rendering the parent commits makes it's easier to compare the PR results to the master so I think that's a good idea!

Rendering the parent commits makes it's easier to compare the PR results to the master so I think that's a good idea!

Merging this since it looks good! We can keep working incrementally on the changes :)

Merging this since it looks good! We can keep working incrementally on the changes :)

Thanks @gs0510!
I did some UI/UX exploration to better understand what exactly we want to present and how. I think this is important to satisfy the different requirements we have for the project. A significant part of the project is user-facing and requires interaction with graphs for basic analysis and a little bit more complex comparisons (i.e., compare a PR to to master or another commit, as suggested by @samoht).
Here are some of the early drafts I created. Things are a bit rough and incomplete, but I left annotations to explain what it all means. There are three main views: Index View (shows the master metrics), PR View and Commit View.
Master View

Annotations

Benchmarks menu - this is only rendered if the project at some point in time produced more than one benchmark results output (as implemented in https://github.com/ocurrent/current-bench/pull/17). For example, we have multiple benchmarks for mirage/irmin, namely mem, layers and pack – these would be presented as separate items in the menu.
Pull requests menu - all of the tracked PRs are listed here. This is identical to what we already have in http://autumn.ocamllabs.io:3030/prs. Links in this menu takes us to the "PR View".
Latest benchmark configuration - the benchmark output contains a config keys which is currently being ignored. This table simply renders the configuration values for the last commit on master.
Metric graphs - all of the metric graphs for each benchmark test are rendered similar separately with a some extra bits of info: test name, value for the last commit, unit (this is currently not included in the JSON output, but should be easy to add), the percentage delta compared to previous commit. Because this view is showing metrics just for master, I wasn't sure if there's anything more useful. The PR view improves this. Also see 6.
Date picker component - this allows to select a specific time interval to be rendered. Currently we are fetching everything, which is already becoming quite slow. By default the last two weeks (with data) should be fetched.
Settings menu - this contains some basic visualisation options: the currently implemented "graph synchronisation" (see https://dygraphs.com/gallery/#g/synchronize for a demonstration); and an option to control the graph values, which I'm considering to implement:

Absolute values - shows the raw values as we currently do;
Relative to previous - for each datapoint this is a percentage compared to the previous commit.
Relative to latest master - for each datapoint this is a percentage compared to the last commit in master. I think this one will be particularly useful for PR views.

Test info tooltip - a description of the test and the metrics. Related to https://github.com/ocurrent/current-bench/issues/16.

PR View

Annotations

"Back to Pull Requests" link - takes us back to "Master View".
Commits menu - the list of all tracked commits for the selected pull request. Links in this menu take us to the "Commit View".
Pull request title - shows the current title of the PR and has a link to the associated GitHub PR.
Parent commits area - the parent commits are greyed out to let the user focus on the commits for the selected PR. This is related to what you said @gs0510.

Commit View

Annotations

"Back to Pull Request" link - takes us back to the parent pull request where the commit was selected.
Commit menu - contains other useful things related to the commit, such as logs (as requested in https://github.com/ocurrent/current-bench/issues/14) and the dockerfile used to run the benchmarks (this might be specially useful when/if we implement https://github.com/ocurrent/current-bench/issues/11).
Commit benchmark configuration - similarly to 3 in "Master View", this shows the config for a given commit.
Metric value cards - since we are looking at a single commit, there isn't a time-series to render as a graph. A simple value is presented with a comparison indicator relative to master (see 5).
Comparison branch selector - the presented commit can be compared against master or any other tag. This is rendered as a percentage in 4. Showing the absolute value for the secondary commit could also be useful.

That's it. Hope this makes sense, let me know if something isn't clear. I think having mocks that encompass the features that we want to see implemented will help us a lot with data modelling and development going forward.
Did I forget something? Did I get something wrong?

Thanks @gs0510!
I did some UI/UX exploration to better understand what exactly we want to present and how. I think this is important to satisfy the different requirements we have for the project. A significant part of the project is user-facing and requires interaction with graphs for basic analysis and a little bit more complex comparisons (i.e., compare a PR to to master or another commit, as suggested by @samoht).
Here are some of the early drafts I created. Things are a bit rough and incomplete, but I left annotations to explain what it all means. There are three main views: Index View (shows the master metrics), PR View and Commit View.
Master View

Annotations

Benchmarks menu - this is only rendered if the project at some point in time produced more than one benchmark results output (as implemented in https://github.com/ocurrent/current-bench/pull/17). For example, we have multiple benchmarks for mirage/irmin, namely mem, layers and pack – these would be presented as separate items in the menu.
Pull requests menu - all of the tracked PRs are listed here. This is identical to what we already have in http://autumn.ocamllabs.io:3030/prs. Links in this menu takes us to the "PR View".
Latest benchmark configuration - the benchmark output contains a config keys which is currently being ignored. This table simply renders the configuration values for the last commit on master.
Metric graphs - all of the metric graphs for each benchmark test are rendered similar separately with a some extra bits of info: test name, value for the last commit, unit (this is currently not included in the JSON output, but should be easy to add), the percentage delta compared to previous commit. Because this view is showing metrics just for master, I wasn't sure if there's anything more useful. The PR view improves this. Also see 6.
Date picker component - this allows to select a specific time interval to be rendered. Currently we are fetching everything, which is already becoming quite slow. By default the last two weeks (with data) should be fetched.
Settings menu - this contains some basic visualisation options: the currently implemented "graph synchronisation" (see https://dygraphs.com/gallery/#g/synchronize for a demonstration); and an option to control the graph values, which I'm considering to implement:

Absolute values - shows the raw values as we currently do;
Relative to previous - for each datapoint this is a percentage compared to the previous commit.
Relative to latest master - for each datapoint this is a percentage compared to the last commit in master. I think this one will be particularly useful for PR views.

Test info tooltip - a description of the test and the metrics. Related to https://github.com/ocurrent/current-bench/issues/16.

PR View

Annotations

"Back to Pull Requests" link - takes us back to "Master View".
Commits menu - the list of all tracked commits for the selected pull request. Links in this menu take us to the "Commit View".
Pull request title - shows the current title of the PR and has a link to the associated GitHub PR.
Parent commits area - the parent commits are greyed out to let the user focus on the commits for the selected PR. This is related to what you said @gs0510.

Commit View

Annotations

"Back to Pull Request" link - takes us back to the parent pull request where the commit was selected.
Commit menu - contains other useful things related to the commit, such as logs (as requested in https://github.com/ocurrent/current-bench/issues/14) and the dockerfile used to run the benchmarks (this might be specially useful when/if we implement https://github.com/ocurrent/current-bench/issues/11).
Commit benchmark configuration - similarly to 3 in "Master View", this shows the config for a given commit.
Metric value cards - since we are looking at a single commit, there isn't a time-series to render as a graph. A simple value is presented with a comparison indicator relative to master (see 5).
Comparison branch selector - the presented commit can be compared against master or any other tag. This is rendered as a percentage in 4. Showing the absolute value for the secondary commit could also be useful.

That's it. Hope this makes sense, let me know if something isn't clear. I think having mocks that encompass the features that we want to see implemented will help us a lot with data modelling and development going forward.
Did I forget something? Did I get something wrong?

Thanks @gs0510!
I did some UI/UX exploration to better understand what exactly we want to present and how. I think this is important to satisfy the different requirements we have for the project. A significant part of the project is user-facing and requires interaction with graphs for basic analysis and a little bit more complex comparisons (i.e., compare a PR to to master or another commit, as suggested by @samoht).
Here are some of the early drafts I created. Things are a bit rough and incomplete, but I left annotations to explain what it all means. There are three main views: Index View (shows the master metrics), PR View and Commit View.
Master View

Annotations

Benchmarks menu - this is only rendered if the project at some point in time produced more than one benchmark results output (as implemented in https://github.com/ocurrent/current-bench/pull/17). For example, we have multiple benchmarks for mirage/irmin, namely mem, layers and pack – these would be presented as separate items in the menu.
Pull requests menu - all of the tracked PRs are listed here. This is identical to what we already have in http://autumn.ocamllabs.io:3030/prs. Links in this menu takes us to the "PR View".
Latest benchmark configuration - the benchmark output contains a config keys which is currently being ignored. This table simply renders the configuration values for the last commit on master.
Metric graphs - all of the metric graphs for each benchmark test are rendered similar separately with a some extra bits of info: test name, value for the last commit, unit (this is currently not included in the JSON output, but should be easy to add), the percentage delta compared to previous commit. Because this view is showing metrics just for master, I wasn't sure if there's anything more useful. The PR view improves this. Also see 6.
Date picker component - this allows to select a specific time interval to be rendered. Currently we are fetching everything, which is already becoming quite slow. By default the last two weeks (with data) should be fetched.
Settings menu - this contains some basic visualisation options: the currently implemented "graph synchronisation" (see https://dygraphs.com/gallery/#g/synchronize for a demonstration); and an option to control the graph values, which I'm considering to implement:

Absolute values - shows the raw values as we currently do;
Relative to previous - for each datapoint this is a percentage compared to the previous commit.
Relative to latest master - for each datapoint this is a percentage compared to the last commit in master. I think this one will be particularly useful for PR views.

Test info tooltip - a description of the test and the metrics. Related to https://github.com/ocurrent/current-bench/issues/16.

PR View

Annotations

"Back to Pull Requests" link - takes us back to "Master View".
Commits menu - the list of all tracked commits for the selected pull request. Links in this menu take us to the "Commit View".
Pull request title - shows the current title of the PR and has a link to the associated GitHub PR.
Parent commits area - the parent commits are greyed out to let the user focus on the commits for the selected PR. This is related to what you said @gs0510.

Commit View

Annotations

"Back to Pull Request" link - takes us back to the parent pull request where the commit was selected.
Commit menu - contains other useful things related to the commit, such as logs (as requested in https://github.com/ocurrent/current-bench/issues/14) and the dockerfile used to run the benchmarks (this might be specially useful when/if we implement https://github.com/ocurrent/current-bench/issues/11).
Commit benchmark configuration - similarly to 3 in "Master View", this shows the config for a given commit.
Metric value cards - since we are looking at a single commit, there isn't a time-series to render as a graph. A simple value is presented with a comparison indicator relative to master (see 5).
Comparison branch selector - the presented commit can be compared against master or any other tag. This is rendered as a percentage in 4. Showing the absolute value for the secondary commit could also be useful.

That's it. Hope this makes sense, let me know if something isn't clear. I think having mocks that encompass the features that we want to see implemented will help us a lot with data modelling and development going forward.
Did I forget something? Did I get something wrong?

Thanks @gs0510!
I did some UI/UX exploration to better understand what exactly we want to present and how. I think this is important to satisfy the different requirements we have for the project. A significant part of the project is user-facing and requires interaction with graphs for basic analysis and a little bit more complex comparisons (i.e., compare a PR to to master or another commit, as suggested by @samoht).
Here are some of the early drafts I created. Things are a bit rough and incomplete, but I left annotations to explain what it all means. There are three main views: Index View (shows the master metrics), PR View and Commit View.
Master View

Annotations

Benchmarks menu - this is only rendered if the project at some point in time produced more than one benchmark results output (as implemented in https://github.com/ocurrent/current-bench/pull/17). For example, we have multiple benchmarks for mirage/irmin, namely mem, layers and pack – these would be presented as separate items in the menu.
Pull requests menu - all of the tracked PRs are listed here. This is identical to what we already have in http://autumn.ocamllabs.io:3030/prs. Links in this menu takes us to the "PR View".
Latest benchmark configuration - the benchmark output contains a config keys which is currently being ignored. This table simply renders the configuration values for the last commit on master.
Metric graphs - all of the metric graphs for each benchmark test are rendered similar separately with a some extra bits of info: test name, value for the last commit, unit (this is currently not included in the JSON output, but should be easy to add), the percentage delta compared to previous commit. Because this view is showing metrics just for master, I wasn't sure if there's anything more useful. The PR view improves this. Also see 6.
Date picker component - this allows to select a specific time interval to be rendered. Currently we are fetching everything, which is already becoming quite slow. By default the last two weeks (with data) should be fetched.
Settings menu - this contains some basic visualisation options: the currently implemented "graph synchronisation" (see https://dygraphs.com/gallery/#g/synchronize for a demonstration); and an option to control the graph values, which I'm considering to implement:

Absolute values - shows the raw values as we currently do;
Relative to previous - for each datapoint this is a percentage compared to the previous commit.
Relative to latest master - for each datapoint this is a percentage compared to the last commit in master. I think this one will be particularly useful for PR views.

Test info tooltip - a description of the test and the metrics. Related to https://github.com/ocurrent/current-bench/issues/16.

PR View

Annotations

"Back to Pull Requests" link - takes us back to "Master View".
Commits menu - the list of all tracked commits for the selected pull request. Links in this menu take us to the "Commit View".
Pull request title - shows the current title of the PR and has a link to the associated GitHub PR.
Parent commits area - the parent commits are greyed out to let the user focus on the commits for the selected PR. This is related to what you said @gs0510.

Commit View

Annotations

"Back to Pull Request" link - takes us back to the parent pull request where the commit was selected.
Commit menu - contains other useful things related to the commit, such as logs (as requested in https://github.com/ocurrent/current-bench/issues/14) and the dockerfile used to run the benchmarks (this might be specially useful when/if we implement https://github.com/ocurrent/current-bench/issues/11).
Commit benchmark configuration - similarly to 3 in "Master View", this shows the config for a given commit.
Metric value cards - since we are looking at a single commit, there isn't a time-series to render as a graph. A simple value is presented with a comparison indicator relative to master (see 5).
Comparison branch selector - the presented commit can be compared against master or any other tag. This is rendered as a percentage in 4. Showing the absolute value for the secondary commit could also be useful.

That's it. Hope this makes sense, let me know if something isn't clear. I think having mocks that encompass the features that we want to see implemented will help us a lot with data modelling and development going forward.
Did I forget something? Did I get something wrong?

This is brilliant @rizo!! I think the config values shouldn't change ideally so maybe we can display it in a sidebar? (i don't have very strong feelings either way).

This is brilliant @rizo!! I think the config values shouldn't change ideally so maybe we can display it in a sidebar? (i don't have very strong feelings either way).


Fix indentation of text

Hello! Thanks for the nice tool.
It's a very minor fix, but I fixed the indentation in the README.
Thank you!

OK, thanks for confirmation!


[test](jdbc catalog) add oceanbase ce jdbc catalog test

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


#65 adding multi tabs to editors

not sure where to put the solution part.. as well as I'm not sure how to test

DO not merge, use for reference


scroll query request

Signed-off-by: Sean Kao<EMAIL_ADDRESS>Description
PoC for SQL scroll request in new query engine
Issues Resolved
#656
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #693 (67b3415) into main (8d9b459) will decrease coverage by 31.98%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##               main     #693       +/-   ##
=============================================
- Coverage     94.74%   62.76%   -31.99%
=============================================
  Files           283       10      -273
  Lines          7676      658     -7018
  Branches        561      119      -442
=============================================
- Hits           7273      413     -6860
+ Misses          349      192      -157
+ Partials         54       53        -1

Flag
Coverage Δ

query-workbench
62.76% <ø> (ø)

sql-engine
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

.../main/java/org/opensearch/sql/planner/Planner.java

.../opensearch/sql/planner/physical/PhysicalPlan.java

...rc/main/java/org/opensearch/sql/storage/Table.java

...opensearch/executor/OpenSearchExecutionEngine.java

...search/executor/protector/ResourceMonitorPlan.java

...ch/sql/opensearch/response/OpenSearchResponse.java

...search/sql/opensearch/storage/OpenSearchIndex.java

...ch/sql/opensearch/storage/OpenSearchIndexScan.java

...ensearch/storage/system/OpenSearchSystemIndex.java

.../opensearch/sql/protocol/response/QueryResult.java

... and 263 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8d9b459...67b3415. Read the comment docs.


Update WindowsBuild.md

Update instructions to build swift bindings for llbuild.  Fix sqlite3 build rule.

Replace this paragraph with a description of your changes and rationale. Provide links to external references/discussions if appropriate.

Resolves SR-NNNN.

@swift-ci please smoke test and merge

@swift-ci please smoke test Linux platform

@swift-ci please smoke test Linux platform


Future work: vectorize using rugged tensors (irregular arrays) and/or Blelloch's flattening

Once vectorization on strictly regular array expressions works (these are expressions that result in regular arrays even after vectorization), generalize it to irregular arrays. Optionally, at the end, check regularity of the result and cast it back as regular arrays (which is close to what Futhark does, sometimes incurring a proof obligation). Alternatively, flatten the irregular tensors early or late and so get regular tensors of lower dimensions (but try not to flatten everything down to vectors, or somehow add a portion of the dimensions back, if possible).
This is needed to expand the language we are able to support (more of the shape-influcencing Ints can be dynamic expressions), Currently, due to emergent irregularity, we need to reject some expressions even in the restricted language (or just not vectorize them, e.g., building tensors of delta expressions instead).

BTW, a way of telling this story is this: we decided to vectorize vector programs and it turned out we need regular tensors to express some results of the vectorization. So we added tensors to the language. In turn, vectorizing those (I'm not sure if proper tensors are needed or if vectors suffice, but the story is better in the former case), we discovered we need irregular (rugged) tensors to express what the vectorization produces.
Now the question arises: if we vectorize irregular tensors (or, possibly, recursively vectorize any of the previous classes to the same effect), are all the results legal rugged tensors or do we need an ever broader class? What class could it even be?
The other question, posed already in the description of the ticket, is whether going back to the simpler classes is feasible and what are the trade-offs involved (it's always possible to go back by full flattening, but then all of the structure is lost, impacting type-safety, performance and possibly more). Another question is how to restrict the relevant classes in order to make them closed under vectorization.

BTW, I've just learned from https://youtu.be/FzfofFPOGOs?t=815 that Accelerate does not have nested arrays (which means rugged tensors, I think), citing hardware limitations, but they have segmented arrays, which probably are rugged tensors represented in the flat way Blelloch proposed. So not only DPH had that representation years ago, but it's found its way to this excellent and modern library.
This may mean rugged tensors are not practical without this flat representation and probably the representation would not be hidden for some practical reasons.

I think, before this ticket, we should attempt recasting the vectorized AD language in Data.Array.Shaped. This would involve writing lots of hard dependently typed code, but the trade-offs are likely to be better than either fully dynamic or fully static assignment of shapes (these would be static, but parameterized on type variables). This is going to be hard, especially without a good (including low dependency footprint) library for sized lists indexed by Nat (I don't know of any).

(I randomly found this issue in the list)

Accelerate does not have nested arrays (which means rugged tensors, I think), citing hardware limitations, but they have segmented arrays, which probably are rugged tensors represented in the flat way Blelloch proposed. So not only DPH had that representation years ago, but it's found its way to this excellent and modern library.

Accelerate has a collection of segmented reductions and segmented scans as primitives, indeed. However, that's the current extent of support of flattening in Accelerate -- there is no automatic flattening of any code. DPH was, if I've understood correctly, much more ambitious in that it actively flattened all the things so the user could be oblivious to the problems of irregular arrays.

I think, before this ticket, we should attempt recasting the vectorized AD language in Data.Array.Shaped.

If I understand correctly, this would mean that the user of horde-ad would also have to write said dependently-typed code, right? Futhark kind of does this, but they have the advantage of being able to tweak the type system to be precisely as they want it. I'd be interested to see how the user experience of such an API in Haskell would be.

Accelerate

Interesting. I fear the flattening resulting in segments is a horror both for the tool writer and user, because it can't be fully abstracted for whatever reasons (probably performance). I wonder if there are any really transparent solutions. Maybe for sparse arrays the rugged aspect can be virtually free and covered with the same machinery as sparsity.

If I understand correctly, this would mean that the user of horde-ad would also have to write said dependently-typed code, right? Futhark kind of does this, but they have the advantage of being able to tweak the type system to be precisely as they want it. I'd be interested to see how the user experience of such an API in Haskell would be.

It's not that bad. See the complete neural networks (with variants) coded here
https://github.com/Mikolaj/horde-ad/blob/612e0c6a06f7981c26ca96693ff905a72f4d9228/README.md
https://github.com/Mikolaj/horde-ad/blob/612e0c6a06f7981c26ca96693ff905a72f4d9228/example/MnistFcnnShaped.hs
https://github.com/Mikolaj/horde-ad/blob/612e0c6a06f7981c26ca96693ff905a72f4d9228/example/MnistRnnShaped.hs
And the current iteration of horde-ad interface has a chance of being more regular and pleasant, if we manage to port it to shaped tensors (I'm certainly thankful we didn't start this with shaped tensors; I'd go mad).
In the worst case you'd do unsafeCoerce (which IIRC, I avoided in these examples at the cost of passing many proxy arguments, which however serve as a more explicit and self-documenting interface and make the errors better).

I fear the flattening resulting in segments is a horror both for the tool writer and user, because it can't be fully abstracted for whatever reasons (probably performance).

That's the reason I'm aware of too, yes. I believe flattening "just works", but gives you pretty bad constant factors if you just apply it naively always.

It's not that bad. See the complete neural networks (with variants) coded here

Interesting. It does seem that apart from having to encode the full shapes in types (which goes without saying), you don't have to do much magic to make it work. Perhaps that changes when the size changes are beyond the powers of ghc-typelits-natnormalise.

In the worst case you'd do unsafeCoerce (which IIRC, I avoided in these examples at the cost of passing many proxy arguments, which however serve as a more explicit and self-documenting interface and make the errors better).

What happens if the user uses unsafeCoerce in their code to make the shapes match up? Horde will then receive a program that has inconsistent typing, in some sense. Will anything break? I feel like that's very dangerous territory -- using unsafeCoerce when calling a library that you know doesn't actually do anything with the types (like orthotope) is "fine" as long as you have your external proof that the coerced shapes are indeed equal. But if the library (in this case, horde-ad) uses the types internally, I wonder if there are subtle ways in which everything can collapse.

What happens if the user uses unsafeCoerce in their code to make the shapes match up? Horde will then receive a program that has inconsistent typing, in some sense.

That's a good point. Perhaps a safer way is to convert to completely unshaped tensors and back (there is an untyped "rank" in both standard horde-ad and even, in minimal form, in the simplified one), which I do in a couple of places. Or flatten and reshape back. Both methods runtime-fail if the number of elements doesn't match and nothing else is required. Such "believe me" terms are fully supported by horde-ad, though see another ticket which lists AstReshape as the hardest nut to crack for vectorization. :)
If, OTOH, horde-ad had differentiable higher-order functions, or anything else that can't be to summarized to "has X elements", then we might be in trouble.


docs: Refactor the README to be more user-oriented

Part of https://github.com/apache/iceberg-rust/issues/429

Looks good. I'm not normally as picky about getting the english exactly correct as long as it is understandable, but for the main Readme.md I think we want it perfect :-)

Thanks for the review. I will address those comments later.

All comments have been resovled. PTAL @liurenjie1024 @Fokko @sdd


[WFLY-11069] test Hibernate51CompatibilityTransformer transformations…

… for o.h.type.Type and o.h.type.AbstractStandardBasicType
https://issues.jboss.org/browse/WFLY-11069

+1 for merging this change in.
The Windows failure doesn't look related to this PR.


Fix confusing error message in VectorAssembler

What changes were proposed in this pull request?
When VectorAssembler encounters a NULL with handleInvalid="error", it throws an exception. This exception, though, has a typo making it confusing. Yet apparently, this same exception for NaN values is fine. Fixed it to look like the right one.

Why are the changes needed?
Encountering this error with such message was very confusing! I hope to save time of fellow engineers by improving it.

Does this PR introduce any user-facing change?
No.

How was this patch tested?
It's just an error message...

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Jenkins test this please

Test build #118982 has started for PR 27709 at commit 8b773ff.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23730/
Test PASSed.

Test build #118982 has finished for PR 27709 at commit 8b773ff.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118982/
Test FAILed.

Jenkins retest this please

Test build #118983 has started for PR 27709 at commit 8b773ff.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/23731/
Test PASSed.

Test build #118983 has finished for PR 27709 at commit 8b773ff.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/118983/
Test PASSed.

Merged to master/3.0/2.4


WIP: attempt to remove pinning to GDAL

Currently, conda won't allow the latest gdal (2.2.0) and rios in the same environment because of this pin. This pin was added to workaround some bug in conda on CircleCI that was selecting a very old gdal version.
In theory rios will work with any version of gdal since it is pure Python and we have made every effort to deal with API changes since about gdal 1.10.
Anyway, this PR will hopefully allow us to see if this is needed anymore.

Hi! This is the friendly automated conda-forge-linting service.
I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

@gillins we should update the pin instead of removing it or the chances for wrong version combination will increase.
(I was waiting on your input to update the gdal pin everywhere but I forgot to ask you :-)

I didn't realize there were other gdal pins elsewhere! They don't seem to be updated automatically (not in https://github.com/conda-forge/conda-forge.github.io/blob/master/scripts/pin_the_slow_way.py) though maybe they should be?
I was hoping to get to a situation where rios would work with whatever is the latest gdal without rebuilding it. Is that possible? And I suppose there will be people wanting to use it with gdal 1.x (we still support this right?).
What have you changed the pin to be elsewhere? 2.2.*?

@gillins we should update the pin instead of removing it or the chances for wrong version combination will increase.
(I was waiting on your input to update the gdal pin everywhere but I forgot to ask you :-)

They don't seem to be updated automatically

I believe that problem was fixed.

What have you changed the pin to be elsewhere? 2.2.*?

Not yet. That is what I wanted to check with you first.

I was hoping to get to a situation where rios would work with whatever is the latest gdal

The problem is that conda solver may get any version/build number when trying to satisfy other packages in the env. By pinning we reduce that to only the build number.
I guess that rios would work fine and maybe the channel preference is all we need here.
I started pinning gdal before the channel preference was implemented due to the large number of gdal dependencies that could come from defaults and break everything.

@ocefpaf happy to go with whatever you think is best. I'd say that C/C++ packages should be re-pinned to 2.2.*.
Was hoping to remove pinning here, but you have way more experience debugging these problems :grin:

I am also inclined to just pin everything to 2.2.*. I'm in transit now but
I can do that once I get back. (Unless someone beats me to it.)

I am also inclined to just pin everything to 2.2.*

Fine by me. I've just updated this PR, merge when you are happy.