r1.3 branch - encountered error attempting build on Pi 2

OS is Ubuntu Mate 16.04 .. cannot quite decipher the problem yet.  Might try the Pi2 specific build line next.
System information
== cat /etc/issue ===============================================
Linux mepi-desktop 4.4.38-v7+ #938 SMP Thu Dec 15 15:22:21 GMT 2016 armv7l armv7l armv7l GNU/Linux

== are we in docker =============================================
No

== compiler =====================================================
c++ (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

== uname -a =====================================================
Linux mypi-desktop 4.4.38-v7+ #938 SMP Thu Dec 15 15:22:21 GMT 2016 armv7l armv7l armv7l GNU/Linux

== check pips ===================================================
numpy (1.11.0)

== check for virtualenv =========================================
False

== tensorflow import ============================================
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named tensorflow

== env ==========================================================
LD_LIBRARY_PATH is unset
DYLD_LIBRARY_PATH is unset

== nvidia-smi ===================================================
tf_env_collect.sh: 105: tf_env_collect.sh: nvidia-smi: not found

== cuda libs  ===================================================

Describe the problem
Attempting to following compilation instructions for RPi, getting error on make -f ...
https://github.com/tensorflow/tensorflow/tree/r1.3/tensorflow/contrib/makefile
tensorflow/contrib/makefile/download_dependencies.sh
sudo apt-get install -y autoconf automake libtool gcc-4.8 g++-4.8
cd tensorflow/contrib/makefile/downloads/protobuf/
./autogen.sh
./configure
make
sudo make install
sudo ldconfig  # refresh shared library cache
cd ../../../../..

make -f tensorflow/contrib/makefile/Makefile HOST_OS=PI TARGET=PI OPTFLAGS="-Os" CXX=g++-4.8

Source code / logs
remote_fused_graph_execute_op.cc:(.text._ZN10tensorflow25RemoteFusedGraphExecuteOpC2EPNS_20OpKernelConstructionE[_ZN10tensorflow25RemoteFusedGraphExecuteOpC5EPNS_20OpKernelConstructionE]+0x198): undefined reference to `google::protobuf::internal::fixed_address_empty_string'
/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a(saved_tensor_slice.pb_text.o): In function `tensorflow::internal::ProtoParseFromScanner(tensorflow::strings::Scanner*, bool, bool, tensorflow::SavedSlice*)':
saved_tensor_slice.pb_text.cc:(.text+0x630): undefined reference to `google::protobuf::internal::fixed_address_empty_string'
/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a(saved_tensor_slice.pb_text.o): In function `tensorflow::internal::ProtoParseFromScanner(tensorflow::strings::Scanner*, bool, bool, tensorflow::SavedSliceMeta*)':
saved_tensor_slice.pb_text.cc:(.text+0x136c): undefined reference to `google::protobuf::internal::fixed_address_empty_string'
/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a(memmapped_file_system.pb_text.o): In function `tensorflow::internal::ProtoParseFromScanner(tensorflow::strings::Scanner*, bool, bool, tensorflow::MemmappedFileSystemDirectoryElement*)':
memmapped_file_system.pb_text.cc:(.text+0x354): undefined reference to `google::protobuf::internal::fixed_address_empty_string'
/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a(saver.pb_text.o): In function `tensorflow::internal::ProtoParseFromScanner(tensorflow::strings::Scanner*, bool, bool, tensorflow::SaverDef*)':
saver.pb_text.cc:(.text+0x698): undefined reference to `google::protobuf::internal::fixed_address_empty_string'
/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/lib/libtensorflow-core.a(cluster.pb_text.o):cluster.pb_text.cc:(.text+0x6e4): more undefined references to `google::protobuf::internal::fixed_address_empty_string' follow
collect2: error: ld returned 1 exit status
tensorflow/contrib/makefile/Makefile:552: recipe for target '/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/bin/benchmark' failed
make: *** [/home/mypi/Dev/git/tensorflow/tensorflow/tensorflow/contrib/makefile/gen/bin/benchmark] Error 1

To build tensorflow on Raspberry Pi, there's a guide for it by @samjabrahams
https://github.com/samjabrahams/tensorflow-on-raspberry-pi/blob/master/GUIDE.md
There are links to compiled whls for tensorflow on his guide for python 2.7 and 3.4. I have also uploaded one for python 3.5 here

Will give these a run through, thank you.  Would a PR with the steps outlined in here be helpful?  Either in master or master & r1.3 ?

@EMCP, are you suggesting a PR for @samjabrahams' instructions or in TensorFlow somewhere?

was waiting to get things working on my end, but yes..
running the instructions that are there now, on a RPi2, seemed to not quite get me there..

The fact that you need to use a USB swap drive makes me think it may not make sense to add those instructions here..  but would be helpful to somehow highlight the way in which some people have shoehorned bazel onto the RPi...
I am asking some questions on those instructions before I would want to write up the steps from there..
My gut is, the script we are leveraging in the official instructions of TF just needs to be updated / verified to work

Assigning @petewarden. You might find his page useful as well
https://petewarden.com/2017/08/20/cross-compiling-tensorflow-for-the-raspberry-pi/

This looks like it can work for me.... only problem is

This compiles for python 2.7, I think I can get by but ideally would want to get a python 3.5 version

the scripts cited in that blog post are not in the r1.3 branch.. so I am guessing they're not really released yet.  adding these instructions then would not make sense for 1.3 branch

In theory, you should just be able to set the following environment variables before building to get Python 3:
export TF_BUILD_CONTAINER_TYPE=CPU
export TF_BUILD_PYTHON_VERSION=PYTHON3
export TF_BUILD_IS_OPT=OPT
export TF_BUILD_IS_PIP=PIP
export USER="${USER:-jenkins}"
tensorflow/tools/ci_build/ci_build.sh PI tensorflow/tools/ci_build/pi/build_raspberry_pi.sh

I haven't tried this myself though, so it may well be trickier than that.
The scripts will be part of the 1.4 release, along with some official 1.4 Pi binaries, but they're also available in the nightlies.

<3 official 1.4 Pi Binaries , but I will give the settings a run and report back here.
Thank you


File upstream issue/pr against Eirini to set fsGroup

#782 is working around the issue of not having write access to PVCs, and not having root access to chmod the mount point.
SUSE/eirini-persi#25 implements this on the persi side, but has to guess as to the correct GID for the running user. It would be much better if runAsGroup and fsGroup would already be set by Eirini, so they don't have to be (guessed) and added by the admission controller.

We haven't seen instances where this is required anymore.


Unable to use Workload Identity Federation and domain-wide delegation to Google Workspace user with googleapiclient upload to Google Drive

TL;DR
I am also struggling with this. I am using the Python googleapiclient library to upload files to Google Drive using both GCP's Workload Identity Provider (WIF) and delegation to my Google Workspace user. The delegation seems to fail and instead, the files are uploaded as the service account (to which I already gave access to the folder), but it then gets stuck later on since it does not have sufficient storage space in it's Google Drive allowance. How can I get this work?
Here's the contents of upload_to_gdrive.py:
import logging
import os
from pathlib import Path
import sys
from googleapiclient.discovery import build
from google.auth import default
from googleapiclient.http import MediaFileUpload
from googleapiclient.errors import HttpError

BATCH_SIZE = 10  # Adjust this value based on your needs
SKIPPED_FOLDERS = {".git"}

logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)

def create_folder_if_not_exists(service, folder_name: str, parent_id: str) -> str:
    query = f"name='{folder_name}' and '{parent_id}' in parents and mimeType='application/vnd.google-apps.folder' and trashed=false"
    results = (
        service.files().list(q=query, spaces="drive", fields="files(id)").execute()
    )
    if results["files"]:
        folder_id = results["files"][0]["id"]
        logger.info(f"Folder '{folder_name}' found with ID: {folder_id}")
        return folder_id
    else:
        folder_metadata = {
            "name": folder_name,
            "mimeType": "application/vnd.google-apps.folder",
            "parents": [parent_id],
        }
        folder = service.files().create(body=folder_metadata, fields="id").execute()
        logger.info(
            f"Folder '{folder_name}' created successfully with ID: {folder.get('id')}"
        )
        return folder.get("id")

def upload_file(service, file_path: str, parent: tuple[str, str]):
    parent_name, parent_id = parent
    file_metadata = {"name": Path(file_path).name, "parents": [parent_id]}
    media = MediaFileUpload(file_path, resumable=True)
    try:
        file = (
            service.files()
            .create(body=file_metadata, media_body=media, fields="id")
            .execute()
        )
        logger.debug(
            f"File '{file_path}' uploaded successfully with ID: {file.get('id')} to folder with name: {parent_name} with ID: {parent_id}"
        )
        return file.get("id")
    except HttpError as error:
        logger.error(f"An error occurred while uploading '{file_path}': {error}")
        raise error

def upload_folder(
    service, folder_path: Path, parent_id: str, target_subfolder: str | None = None
):
    target_parent_id = (
        create_folder_if_not_exists(service, target_subfolder, parent_id)
        if target_subfolder
        else parent_id
    )
    folder_path_to_id = {
        Path("."): target_parent_id
    }  # Caching the root folder as the parent ID

    for root, dirs, files in folder_path.walk():
        if any(folder in str(root) for folder in SKIPPED_FOLDERS):
            continue
        # Cache folder paths to prevent redundant folder creation
        for dir_name in dirs:
            dir_path = Path(root) / dir_name
            relative_path = dir_path.relative_to(folder_path)

            if relative_path not in folder_path_to_id:
                folder_id = create_folder_if_not_exists(service, dir_name, parent_id)
                folder_path_to_id[relative_path] = folder_id

        # Process files in batches
        file_batch = []
        for file in files:
            file_path = root / file
            rel_folder_path = file_path.parent.relative_to(folder_path)
            folder_id = folder_path_to_id.get(rel_folder_path, parent_id)
            file_batch.append((file_path, (rel_folder_path, folder_id)))

            if len(file_batch) == BATCH_SIZE:
                upload_batch(service, file_batch)
                file_batch = []

        if file_batch:
            upload_batch(service, file_batch)

def upload_batch(service, file_batch: list[tuple[str, tuple[str, str]]]):
    for file_path, (parent_name, parent_id) in file_batch:
        upload_file(service, file_path, (parent_name, parent_id))

def main(folder_path: str, target_subfolder: str | None = None):
    credentials, project = default()

    service = build("drive", "v3", credentials=credentials)
    folder_id = os.environ.get("GOOGLE_DRIVE_FOLDER_ID")
    if not folder_id:
        raise ValueError("GOOGLE_DRIVE_FOLDER_ID environment variable is not set")
    upload_folder(service, Path(folder_path), folder_id, target_subfolder)

if __name__ == "__main__":
    if len(sys.argv) < 2:
        logger.warn("Usage: python upload_to_drive.py <folder_path>")
        sys.exit(1)
    folder_path = sys.argv[1]
    target_subfolder = sys.argv[2] if len(sys.argv) == 3 else None
    if not Path(folder_path).is_dir():
        logger.error(f"Error: '{folder_path}' is not a valid directory.")
        sys.exit(1)
    main(folder_path, target_subfolder)

Expected behavior
The files should be uploaded as my Google Workspace user
Observed behavior
The files are uploaded as the Service Account user until the user hits a storage limit (I have given the service account permissions to the folder and I am uploading large files)
Action YAML
permissions:
  id-token: write # This is required for requesting the JWT
  contents: read # This is required for actions/checkout
jobs:
  Deploy:
    runs-on: ubuntu-latest-m
    steps:
      - name: Git clone the repository
        uses: actions/checkout@v4
      - name: Google auth
        id: 'auth'
        uses: google-github-actions/auth@v2
        with:
          token_format: 'access_token'
          workload_identity_provider: '${{ secrets.GCP_WIF_PROVIDER }}'
          service_account: '${{ secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT }}'
          access_token_lifetime: 1800s
          access_token_scopes: https://www.googleapis.com/auth/drive
          access_token_subject: '${{ vars.GOOGLE_DRIVE_SUBJECT_ACCOUNT }}'
          delegates: '${{ secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT }}'

      - name: Print the credentials file path
        run: |
          echo "Credentials file path: ${{ steps.auth.outputs.credentials_file_path }}"
          cat ${{ steps.auth.outputs.credentials_file_path }}

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.x'
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client

      - name: Upload to Google Drive
        env:
          GOOGLE_DRIVE_FOLDER_ID: ${{ secrets.GOOGLE_DRIVE_FOLDER_ID }}
          GOOGLE_APPLICATION_CREDENTIALS: ${{ steps.auth.outputs.credentials_file_path }}
        run: |
          python .github/workflows/upload_to_gdrive.py my-folder target-folder-location

Log output
##[debug]Evaluating condition for step: 'Google auth'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Google auth
##[debug]Register post job cleanup for action: google-github-actions/auth@v2
##[debug]Loading inputs
##[debug]Evaluating: secrets.GCP_WIF_PROVIDER
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GCP_WIF_PROVIDER'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Evaluating: secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GOOGLE_DRIVE_SERVICE_ACCOUNT'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Evaluating: vars.GOOGLE_DRIVE_SUBJECT_ACCOUNT
##[debug]Evaluating Index:
##[debug]..Evaluating vars:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GOOGLE_DRIVE_SUBJECT_ACCOUNT'
##[debug]=><EMAIL_ADDRESS>##[debug]Result<EMAIL_ADDRESS>##[debug]Evaluating: secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT
##[debug]Evaluating Index:
##[debug]..Evaluating secrets:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'GOOGLE_DRIVE_SERVICE_ACCOUNT'
##[debug]=> '***'
##[debug]Result: '***'
##[debug]Loading env
Run google-github-actions/auth@v2
##[debug]Using workload identity provider "***"
##[debug]ID token url is https://pipelinesghubeus6.actions.githubusercontent.com/SKne7tZ5N7wvk8SqLMNClC7Djusf78ztXROgwIwWBbKxy0MekT/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/Actions/plans/f1a39477-9491-423a-882e-dfbe5ce25679/jobs/e07742bd-189a-5079-918b-43f8b2f94b89/idtoken?api-version=2.0&audience=https%3A%2F%2Fiam.googleapis.com%2F***
::add-mask::***
##[debug]WorkloadIdentityFederationClient: Computed audience, //iam.googleapis.com/***
##[debug]Creating credentials file
##[debug]WorkloadIdentityFederationClient.createCredentialsFile: Enabling service account impersonation via https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/***:generateAccessToken
##[debug]WorkloadIdentityFederationClient.createCredentialsFile: Creating credentials, {
##[debug]  "outputPath": "/home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json"
##[debug]}
Created credentials file at "/home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json"
##[debug]WorkloadIdentityFederationClient.getToken: Built request, {
##[debug]  "method": "POST",
##[debug]  "path": "https://sts.googleapis.com/v1/token",
##[debug]  "body": {
##[debug]    "audience": "//iam.googleapis.com/***",
##[debug]    "grantType": "urn:ietf:params:oauth:grant-type:token-exchange",
##[debug]    "requestedTokenType": "urn:ietf:params:oauth:token-type:access_token",
##[debug]    "scope": "https://www.googleapis.com/auth/cloud-platform",
##[debug]    "subjectTokenType": "urn:ietf:params:oauth:token-type:jwt",
##[debug]    "subjectToken": "***"
##[debug]  }
##[debug]}
##[debug]Successfully generated auth token
::add-mask::***
##[debug]Creating access token
##[debug]Using Domain-Wide Delegation flow
##[debug]WorkloadIdentityFederationClient.getToken: Using cached token, {
##[debug]  "now":<PHONE_NUMBER>601,
##[debug]  "cachedAt":<PHONE_NUMBER>495
##[debug]}
##[debug]WorkloadIdentityFederationClient.signJWT: Built request, {
##[debug]  "method": "POST",
##[debug]  "path": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/***:signJwt",
##[debug]  "headers": {
##[debug]    "Authorization": "***"
##[debug]  },
##[debug]  "body": {
##[debug]    "payload": "{\"iss\":\"***\",\"aud\":\"https://oauth2.googleapis.com/token\",\"iat\":1725886770,\"exp\":1725888570,\"sub\":\"my-google-workspace-user@my-workspace.com\",\"scope\":\"https://www.googleapis.com/auth/drive\"}"
##[debug]  }
##[debug]}
##[debug]IAMCredentialsClient.generateDomainWideDelegationAccessToken: Built request, {
##[debug]  "method": "POST",
##[debug]  "path": "https://oauth2.googleapis.com/token",
##[debug]  "headers": {
##[debug]    "Accept": "application/json",
##[debug]    "Content-Type": "application/x-www-form-urlencoded"
##[debug]  },
##[debug]  "body": "grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=***"
##[debug]}
::add-mask::***
##[debug]Node Action run completed with exit code 0
##[debug]CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE='/home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json'
##[debug]GOOGLE_APPLICATION_CREDENTIALS='/home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json'
##[debug]GOOGLE_GHA_CREDS_PATH='/home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json'
##[debug]CLOUDSDK_CORE_PROJECT='hirundo-mvp-dev'
##[debug]CLOUDSDK_PROJECT='hirundo-mvp-dev'
##[debug]GCLOUD_PROJECT='hirundo-mvp-dev'
##[debug]GCP_PROJECT='hirundo-mvp-dev'
##[debug]GOOGLE_CLOUD_PROJECT='hirundo-mvp-dev'
##[debug]Set output credentials_file_path = /home/runner/work/infrastructure/infrastructure/gha-creds-be2508996412c6e7.json
##[debug]Set output project_id = hirundo-mvp-dev
##[debug]Set output auth_token = ***
##[debug]Set output access_token = ***
##[debug]Finishing: Google auth
...
googleapiclient.errors.ResumableUploadError: <HttpError 403 when requesting None returned "The user's Drive storage quota has been exceeded.". Details: "[{'message': "The user's Drive storage quota has been exceeded.", 'domain': 'usageLimits', 'reason': 'storageQuotaExceeded'}]">

Additional information
No response

Hi @benglewis - the error you're getting is because of quota (number of requests per second) to the Drive API. From the debug logs, the domain-wide delegation token was successfully created. To use the token, you need to pull the generated access_token from the auth action outputs. There's no mechanism for ADC to declare a DWD scope; only the generated token will have the DWD scope.

Hi @sethvargo ,
I would greatly appreciate if you can explain with an example how I can "pull the generated access_token from the auth action outputs". I tried multiple times to load the credentials JSON output, both from the file path and environment variable. Every time I got different weird errors (like the JSON being invalid and not credentials for a service account, for example). I would really appreciate just a little more guidance and some explanation of how I can use that access_token properly.
Thank you in advance :)

@sethvargo I figured it out...
Here are the needed changes:

Adjust the Python script:

from google.oauth2.credentials import Credentials
...
def main(folder_path: str, target_subfolder: str | None = None):
    credentials = Credentials(os.environ["GOOGLE_APPLICATION_ACCESS_TOKEN"])
    service = build("drive", "v3", credentials=credentials)

Adjust the GitHub Actions workflow:

...
- name: Upload to Google Drive
        env:
          GOOGLE_DRIVE_FOLDER_ID: ${{ secrets.GOOGLE_DRIVE_FOLDER_ID }}
          GOOGLE_APPLICATION_ACCESS_TOKEN: ${{ steps.auth.outputs.access_token }}
        run: |
          python .github/workflows/upload_to_gdrive.py  my-folder target-folder-location
...

That is it :)

It's an output of the auth action: https://github.com/google-github-actions/auth?tab=readme-ov-file#outputs

Right. The issue I faced was understanding how to use it together with the Python library. My above code shows how I was able to get it to work 🙂


Fix: Update rootdisksize detail on restore VM

Description
This PR fixes the below issue:

Register two VM templates, small and large.
Deploy VM using smaller template while overriding the rootdisksize
Start VM if not started
Stop VM
Reinstall VM to larger VM template
Observe new root is the size of old volume. This is happening because the rootdisksize detail for VM overrides the template's size.

Types of changes

[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] New feature (non-breaking change which adds functionality)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] Enhancement (improves an existing feature and functionality)
[ ] Cleanup (Code refactoring and cleanup, that may add test cases)
[ ] build/CI

Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale

[ ] Major
[x] Minor

Bug Severity

[ ] BLOCKER
[ ] Critical
[ ] Major
[x] Minor
[ ] Trivial

Screenshots (if appropriate):
How Has This Been Tested?

How did you try to break this feature and the system with this change?

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 4.31%. Comparing base (e409c6d) to head (dc41e2b).

Additional details and impacted files
@@             Coverage Diff              @@
##               4.19   #8981       +/-   ##
============================================
- Coverage     14.96%   4.31%   -10.65%
============================================
  Files          5373     363     -5010
  Lines        468989   29171   -439818
  Branches      61009    5089    -55920
============================================
- Hits          70191    1260    -68931
+ Misses       391019   27770   -363249
+ Partials       7779     141     -7638

Flag
Coverage Δ

uitests
4.31% <ø> (ø)

unittests
?

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@blueorangutan package

@sureshanaparti a [SL] Jenkins job has been kicked to build packages. It will be bundled with  KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress.

Packaging result [SF]: ✔ el7 ✔ el8 ✔ el9 ✔ debian ✔ suse15. SL-JID 9433

@blueorangutan test keepEnv

@borisstoyanov a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

[SF] Trillian test result (tid-10049)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 44709 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr8981-t10049-kvm-centos7.zip
Smoke tests completed. 129 look OK, 1 have errors, 0 did not run
Only failed and skipped tests results shown below:

Test
Result
Time (s)
Test File

test_01_events_resource
Error
411.42
test_events_resource.py


How can a context field be delete with transformers?

For example, I have this message from azure -
{
        "specversion": "1.0",
        "id": "9ef1cd85-101e-004b-2e44-2e73ab0663b7",
        "source": "/subscriptions/<subscription>/resourceGroups/triggermesh/providers/Microsoft.Storage/storageAccounts/triggermeshblob",
        "type": "Microsoft.Storage.BlobCreated",
        "subject": "/blobServices/default/containers/kallie/blobs/somepicture.jpeg",
        "datacontenttype": "application/json",
        "time": "2023-01-22T09:32:45.8761294Z",
        "data": {...}
    }

Ultimately I want to remove ALL the fields from the context (i.e all the fields that are not data - specversion, id, source...), how can this be achieved?
This does not work and doesn't remove the field -
context:
- operation: delete
  paths:
  - key: specversion

Which Target are you using ?

Kafka target

yes, it worked thank you!
I am using Kubernetes YAML.


Update Readme on new machine script and hostname

[ ] Update Readme on new machine script and hostname

Closed per https://github.com/iancleary/ansible-desktop/pull/128/commits/c7fb4c88df6d9445fb170636d602c08b1b5e374c within https://github.com/iancleary/ansible-desktop/pull/128


CCRenderTexture: fetching Texture not working

From the forums: http://forums.xamarin.com/discussion/31704/problem-creating-a-ccsprite-out-of-a-ccdrawnode#latest

This should be a non issue.  The user example was trying to draw to a render texture using a CCDrawNode which is actually part of the Scene Graph and follows flow of the Scene Graph.
To actually draw to the RenderTarget that is created you will need to use the CCDrawingPrimitives which draws to the GraphicsDevice which the RenderTarget is attached to example following:
        // create render texture containing draw node
        CCRenderTexture rtm = new CCRenderTexture(new CCSize(200.0f, 200.0f), new CCSize(200.0f, 200.0f), CCSurfaceFormat.Color, CCDepthFormat.Depth24Stencil8);

        rtm.Begin();

        CCDrawingPrimitives.Begin();

        CCDrawingPrimitives.DrawColor = new CCColor4B(255, 255, 255, 255);
        CCDrawingPrimitives.DrawSolidCircle(new CCPoint(100.0f, 100.0f), 75.0f, 0, 20);

        CCDrawingPrimitives.End();

        rtm.End();

        // create sprite from render texture
        CCSprite circleSprite = new CCSprite(rtm.Texture);
        circleSprite.Position = new CCPoint(200.0f, 200.0f);

        AddChild(circleSprite);

We will wait to hear back from the user before closing this but to me this is as designed.

closed via https://github.com/mono/CocosSharp/commit/f1eb03c54fbc1eb15456ed12e93cfc158055f01f


It appears that underscores as keys in a record break the encode function

# Record Test
types = Types.Record({
        'to':Types.Text,
        'fee': Types.Int,
        'memo':Types.Nat64,
        'fromsubaccount':Types.Text,
        #'created_at_time':Types.Record({'timestamp_nanos':Types.Nat64}),
        'amount':Types.Nat64,
    }
)
vals = {
        'to':wallet_address_to_send_ICP,
        'fee': 10000,
        'memo':0,
        'fromsubaccount':'asdf',#convert_wallet_address_to_bytes_array(purchasing_wallet_address)],
        #'created_at_time':[],
        'amount':price,
}
params = [
    {'type': types, 'value': vals}
]
params = encode(params)

The code above works, but if you include an underscore in "from_subaccount" like so, then it gives you an error. I think the underscore is being parsed incorrectly.

Fixed! Now, underscore is supported now!
BTW, You can refer to the example directory to see all types of usage

YES! This is amazing. Just did my first ICP transfer with the ledger canister straight from ic-py. Feels pretty awesome. Here's my working code if you want to piggy back off of it for examples. Or more likely if you want to point out anything I'm doing in an inefficient way. :)
from ic.identity import Identity
from ic.client import Client
from ic.agent import Agent
from ic import Principal
from ic.candid import encode, decode, Types
import time
import random

# you can download a .pem file from plug wallet
# please keep your private key safe
with open('path/to/.pem','r') as f:
    private_key_1 = f.read()

# load private key as identity
i1 = Identity.from_pem(private_key_1)

# initialize client and create an agent
client = Client(url = "https://ic0.app")
agent = Agent(i1, client)

# custom function to convert wallet address to bytes array
# there has to be a better way, but couldn't figure it out
# so I just created my own function to do it
def convert_wallet_address_to_bytes_array(wallet_address):
    final_text = []
    for x in range(int(len(wallet_address) / 2)):
        my_hexdata = wallet_address[x*2:x*2+2]
        final_text.append(int(my_hexdata,16))
    return final_text

# the ICP ledger canister on mainnet
# WARNING this will transfer real ICP from your plug wallet
ledger_canister_id = 'ryjl3-tyaaa-aaaaa-aaaba-cai'

# the types expected from a ledger canister transfer call
types = Types.Record({
        'to':Types.Vec(Types.Nat8),
        'fee': Types.Record({'e8s':Types.Nat64}),
        'memo':Types.Nat64,
        'from_subaccount':Types.Opt(Types.Vec(Types.Nat8)),
        'created_at_time':Types.Opt(Types.Record({'timestamp_nanos':Types.Nat64})),
        'amount':Types.Record({'e8s':Types.Nat64})
    }
)

# the values expected from a ledger canister transfer call
values = {
        'to':convert_wallet_address_to_bytes_array(wallet_address_to_send_ICP),
        'fee': {'e8s':int(0.0001E8)},
        'memo':0,
        'from_subaccount':[],
        'created_at_time':[{'timestamp_nanos':time.time_ns()}],
        'amount':{'e8s':price},
}

# the correct structure for parameters
params = [
    {'type': types, 'value': values}
]

# encoding parameters
params = encode(params)

# final update call to the ledger canister to transfer from
# your identity (wallet address) to the address specified in the request
result = agent.update_raw(ledger_canister_id, 'transfer', params)

PR is welcome. You can submit this code as an example and put it in example directory.


Feature request: Recursive testing of a given directory

It would be great if you could provide a -r option to the command-line nodeunit program, so that the directory you give it is recursively searched and all tests in all subdirectories are ran. Thanks!

resolved via #308


Enable using proof heirarchy from depends clauses

~Blocked on: https://github.com/runtimeverification/pyk/pull/663~
~Blocked on: https://github.com/runtimeverification/evm-semantics/pull/2098~
~Blocked on: https://github.com/runtimeverification/evm-semantics/pull/2092~
This takes the output of APRProof.from_spec_modules and discharges them in order. In particular:

Take advantage of the existing topological sort of APRProof.from_spec_modules
Do not discharge super proofs until the subproofs are discharged.
Removes two now unused helpers: claim_dependency_graph and ensure_k_sequence_on_k_cell.

Closing due to age.


TypeError: Cannot read property 'name' of undefined

Please assume that the following statement is executing right after all models and assosications are defined:
const context = createContext(fastify.sequelize);

TypeError: Cannot read property 'name' of undefined
    at sequelize.modelManager.forEachModel.Model (\foundation\node_modules\dataloader-sequelize\lib\index.js:552:19)
    at ModelManager.forEachModel (\foundation\node_modules\sequelize\lib\model-manager.js:91:7)
    at createContext (\foundation\node_modules\dataloader-sequelize\lib\index.js:551:26)
    at main (\foundation\api\node\plugins\fastify\sequelize-dataloader.js:16:33)
    at Plugin.exec (\foundation\node_modules\avvio\plugin.js:82:17)
    at Boot.loadPlugin (\foundation\node_modules\avvio\plugin.js:166:10)
    at Task.release (\foundation\node_modules\fastq\queue.js:127:16)
    at worked (\foundation\node_modules\fastq\queue.js:169:10)
    at toLoad.finish (\foundation\node_modules\avvio\plugin.js:169:7)
    at done (\foundation\node_modules\avvio\plugin.js:127:5)
    at check (\foundation\node_modules\avvio\plugin.js:138:7)
    at processTicksAndRejections (internal/process/next_tick.js:74:9)

{
  "dataloader": "^1.4.0",
  "dataloader-sequelize": "^1.7.8",
  "sequelize": "^4.42.0",
}

Please show the code that produces this error

Executing the following statement throws different error:
fastify.decorate('seqContext', createContext(fastify.db.Sequelize));

Undefined, it means the fastify.db is the right one:
Executing (default): SELECT 1+1 AS result
Connection has been established successfully.
TypeError: Cannot read property 'prototype' of undefined
    at createContext (\foundation\node_modules\dataloader-sequelize\lib\index.js:540:61)
    at main (\foundation\api\node\plugins\fastify\sequelize-dataloader.js:16:33)
    at Plugin.exec (\foundation\node_modules\avvio\plugin.js:82:17)
    at Boot.loadPlugin (\foundation\node_modules\avvio\plugin.js:166:10)
    at Task.release (\foundation\node_modules\fastq\queue.js:127:16)
    at worked (\foundation\node_modules\fastq\queue.js:169:10)
    at toLoad.finish (\foundation\node_modules\avvio\plugin.js:169:7)
    at done (\foundation\node_modules\avvio\plugin.js:127:5)
    at check (\foundation\node_modules\avvio\plugin.js:138:7)
    at processTicksAndRejections (internal/process/next_tick.js:74:9)
[nodemon] app crashed - waiting for file changes before starting...

foundation\node_modules\dataloader-sequelize\lib\index.js:552
  loaders[Model.name] = {
              ^

I have the same issue

This worked for me
I replaced
sequelize.modelManager.forEachModel(model => ...
with
Object.values(sequelize.models).forEach(model => ...

What versions of Sequelize @jcfinnerup? Was the modelManager removed recently?

I use sequelize v5, however i tried with sequelize v4 and had the same issue.
Sequelize seems to keep a reference to modelManager in their latest docs, and i've tried using the modelManager.forEachModel which seems to work correctly.
It seems that forEachModel iterates over the model names directly, and not objects with an attribute name.
Maybe it has something to do with how i Initialise my models, but its pretty weird.

Update
I found the issue in my case. I had several models which had a reference to other models which had not been initialized/defined. This didn't affect the sequelize.models object, but did cause the ModelManager to return undefined. Hence the error "cannot read name of undefined". I removed the references and everything worked fine.


2.0b1 bshdoc.bsh xsl path wrong and path separator hardcoded

The stylesheet import generated by scripts/bshdoc.bsh
needed to be changed in order for me to view the
bshcommands-bshdoc.xml file in a browser (firefox 0.9):
print("<?xml-stylesheet type="text/xsl"
href="xsl/bshcommands.xsl"?>");
The Name generated by bshdoc should use
java.io.File.separator instead of hardcoding "/".
Without this the name becomes an absolutie path when
run under winders.
submitted by: m_sample at canada.com
Reported by: *anonymous
Original Ticket: "beanshell/bugs/176":https://sourceforge.net/p/beanshell/bugs/176

documentation out of scope for 3.0 release


Installing tip on darwin

Hey,
How do I install go tip on Mac?
I have tried several options but it says that 'tip' does not exist.
I am probably doing something wrong as it seems like it works on travis.
Thanks,
Shaked

Ok I figured it out. It seems like I had to install go 1.4 via gimme otherwise it won't find the repository.


Is it possible to export the feature engineering pipeline and model as tensorflow code?

I'd like to export the feature engineering steps and model as python code, so I can make further customization. It will be great if we can automatically generate the training and prediction code.

Is it possible with current code base?
Will it be considered in future development?

Thanks and great work!

#hhbyyh ludwig is all written in python, so you have all the code you want and you can change whatever you want :) Maybe I'm not getting your question really,

Thanks for the response. @w4nderlust
Suppose I trained the model with:
ludwig train --data_csv path/to/file.csv --model_definition "{input_features: [{name: doc_text, type: text}], output_features: [{name: class, type: category}]}"
After the training I got the saved model and training logs. Yet I want to add some customized operation to feature engineering step or model, the best way in my mind would be to export the training code from ludwig add add my change to it. That means besides the trained model, I would also like to have the code that generates the model (preferably in TF API). Is it possible now?

I see. At the moment and for the forseable future, the preprocessing is performed in python so there is no way to export it as tf code. if you want to modify it, I suggest you to modify the python code directly, check ludwig/data/preprocessing.py to see what happens when Ludwig does the preprocessing and play around with it. Consider that each feature has it's own preprocessing that you can find in the BaseFeature object in each /ludwig/feature*_feature.py module.
One thing i would suggest you anyway is to avoid changing the preprocessing after you trained a model, as discrepancies in the pipeline you use for prediction may result in poor results. If you want to change the pipeline do it before training, and then keep the same one when predicting. Also consider that if you call the LudwigModel.predict() method from the programmatic API, the pipeline of the trained Ludwig model is applied automatically for you.
Marking this as a feature request, but also as a wontfix, at least for now. Feel free to keep asking questions if you have more.

Thanks a lot for providing the answer. I'll close the issue now.


Issue when using magento recipe

Hi,
I'm testing your UI, seems cool for now.
I'm tryng to deploy a magento website so i used the basic magento recipe, but when i want to deploy i have an error :
ErrorException in DeploymentForm.php line 63: Array to string conversion in DeploymentForm.php line 63 at HandleExceptions->handleError('8', 'Array to string conversion', '/home/xxxxxxxx/webloyer/app/Services/Form/Deployment/DeploymentForm.php', '63', array('input' => array('_token' => 'jnVHgzYJTUQ2hIaO828ym2lWVYsZCuo6QfCULeqr', 'task' => 'deploy', 'status' => null, 'message' => null, 'project_id' => '1', 'user_id' => '1'), 'deployment' => object(Deployment))) in DeploymentForm.php line 63 at DeploymentForm->save(array('_token' => 'jnVHgzYJTUQ2hIaO828ym2lWVYsZCuo6QfCULeqr', 'task' => 'deploy', 'status' => null, 'message' => null, 'project_id' => '1', 'user_id' => '1')) in DeploymentsController.php line 72 at DeploymentsController->store(object(Request), object(Project)) at call_user_func_array(array(object(DeploymentsController), 'store'), array(object(Request), 'projects' => object(Project))) in compiled.php line 8899
I'm not sure when i did something wrong (server configuration / deployment configuration ?)
Here is my server config :
production:
host: XXXXXX.com
user: XXXXXX
password: XXXXXX
stage: production
deploy_path: /home/www/XXXXXX/magento
And my deployment config
Project Name    XXXXXXXX - PROD
Recipe  deployer-magento-recipe
Server  XXXXXXXX
Repository URL  https://gitlab.com/XXXXXXXX/XXXXXXXX.git
Stage   production
Deploy Path
E-Mail Notification Recipient
If any idea, i'll take it :-)
Thx

I've just see that i didn't declare servers correctly.
My mistake ! Sorry for posting too quickly;

Hi,
What is the version of PHP you are using?
At the moment, Webloyer has not been tested with PHP 7.
Therefore, if you are using PHP 7, Webloyer may not work properly.

Hi ngmy,
You're right, i'm on PHP7.
I will try on a PHP5 server then, thx !

Support for PHP 7 is planned for the future.
Please give me some time.

Should be fixed with #61

I have added support PHP 7.
Please try the version 0.40.0.

I have added support PHP 7.
Please try the version 0.40.0.


Update Solidus dependency version

We want to support 3.0 with this PR. Also, it's using some code
from solidus_support that was introduced in 0.8.1, so this
should be the minimum version required.
This PR is dependent on Solidus Tracking #30 and will not work until that PR is merged.

Taking care of solidus_tracking, I think we need to release a new version to see green specs here.

No, sorry, the release is not required because solidus_tracking is included in the Gemfile taking the last GitHub version. I think you can safely rebase here and it should pass, thanks!


Support VNET Integration

Allow users to host the ACA cluster in a VNET behind an Application Gateway or Azure Front Door.

I believe this will be mostly covered with deploying application in the ACA landing zone. Atleast using Application gateway bit is already covered in the PR which is raised, but we can look into adding support for front door as well.
Then we need to make the guidance part of the workshop content itself.


Component Conversion Examples

Sorry! I actually meant to put this up against my fork. Apparently I forgot to change the target repo.
But now I'm actually really glad I messed this up!

We could have both the enum and int versions of the method, and reconcile the information in the ComponentProcessor to provide a normal form that is consumed by the option blocks.

This makes sense to me! I hadn't throught about needing backwards compatibility wrt the java-side method definitions.
It would also make handling the Blockly -> Java type conversions. Ie if the block accepted numbers (for backwards compatibility) but the Java function only accepted a Direction we would need to convert it at some point. But if we also have a Java function that accepts an int, no conversion is needed (assuming kawa handles overloads gracefully).
But I don't know how we would handle getters then because you can't overload a getter.
The getters need to be typed as ints on the Blockly side for backwards compatibility. But we want the dropdown blocks to be typed as enums (eg Direction) on the blockly side so that you can't connect the wrong dropdown block to the wrong input. But we also need to be able to compare the getter to the dropdown block, or a variable assigned to a dropdown block...
I need to figure out a better way to visualize how we want the typing system to work. Once I come up with something I'll reply again.

Ok I think I've got it figured out.
So current components will get overloads for their methods, events, and setters added to the Java definitions. But future components will only accept enums (e.g. Direction, Alignment, etc).
Old-style methods, events, and setters will accept either their old types, or the new enums. (Ideally kawa would be able to handle this automatically) Old-style getters will return their old types, since we can't overload getters.
New-style methods, events, and setters will /only/ accept enums. New-style getters will return enums.
Dropdowns will return enums.
If we need to compare an old-style getter with a dropdown the enum returned by the dropdown can be easily converted to a simple type (string, int, etc). I'm not sure who would handle this conversion though (block generators? runtime.scm?).
Does that sound like what you were thinking @ewpatton ?

@BeksOmega

So current components will get overloads for their methods, events, and setters added to the Java definitions. But future components will only accept enums (e.g. Direction, Alignment, etc).

Correct.

Old-style method, event, and setter blocks will accept either their old types, or the new enums. (Ideally kawa would be able to handle this automatically) Old-style getter blocks will return their old types, since we can't overload getters.

We will need to add a path to coerce-args in runtime.scm that can coerce a number to the enum type if the enum type is present on the block, otherwise fall back to the numeric value.

If we need to compare an old-style getter block with a dropdown block the enum returned by the dropdown can be easily converted to a simple type (string, int, etc). I'm not sure who would handle this conversion though (block generators? runtime.scm?).

Likely what we would want to do is update yail-atomic-equal? in runtime.scm to handle the case where one of the operands is an enum and another is a number-like (integer, or string coercible to integer).


Working with int64 timestamps (JTD)

I have a timestamp that I need to validate, and in my JSON it looks like this: "postedOn":<PHONE_NUMBER>000,.
I'm using JTD with Typescript. The data type of this timestamp would be int64, however it looks like this type is not implemented in ajv.
I found a package that implements it - https://ajv.js.org/packages/ajv-formats.html, however it turns out I can't use formats keyword with JTD.
I'm stuck, and I can't find anything helpful online. I believe int64 is a very common use-case, and I'm surprised to not find any clear guidance on that.

int64 is not recommended to use for Internet messages for compatibility reasons. In fact, JavaScript itself does not support int64, the maximum integer it can represent is 2^53 – JTD spec has a reference to a relevant RFC.
There is an option int32range for JTD validation that removes range checking though: https://ajv.js.org/options.html#int32range
JTD supports type: timestamp though - why can't you use this one instead?
"format" (or any other non-standard JTD keyword) can be used with JTD inside metadata member – it has to be added to the instance using addKeyword method, e.g. "format" definition is here: https://github.com/ajv-validator/ajv/blob/master/lib/vocabularies/format/format.ts (you have to import compiled file from "dist" folder though).

Thank you for your help. I have briefly looked through formats allowed by rfc3339 and figured out that they don't accept timestamps in number-of-ticks-since format, therefore I didn't want to use timestamp type.
However, I just read the whole paper and it looks like this is the endorsed standard for the Internet, so I guess I'll have redesign the system, so it produces rf3339 supported timestamps.
Unfortunately, it doesn't accept milliseconds (only fractions of a second), but luckily I don't need them in my case.

It does accept milliseconds after the decimal point in seconds parts.
Using the number is fine too, if you don’t need milliseconds 2^32 seconds will last you 49000 years, and if you do - you could either store it as a decimal string or use JS integer limit where 2^53 will last over 200k years, so all should be fine.


visualization of data

Description
visualization of data from csv file as a graph format
Domain
Datascience with Python
Type of Contribution
Documentation
Code of Conduct

[X] I follow Contributing Guidelines & Code of conduct of this project.

/assign

serial no.-116
batch-4
name-ashutosh singh

Don't create issues without informing. @ashutoshlovpry


create monsters

create a list of monster to be used in explorations
structure:
  id?: string
  name: string
  level: number
  damage: number
  hp: number
  shield: number
  givedXp: number
  givedGold: number

Firebase DB was released with some new monsters.
Please validate.

Same thing in https://github.com/lgmagalhaes88/AdventureRPG/issues/20#issuecomment-429300561


Why not RocksDB for streaming storage?

https://materialize.com/blog-rocksdb/

RocksDB Primer

Functions: point lookups, inserts, and range scans under strong isolation guarantees at high concurrent performance
Users: Main OLTP engines (CockroachDB, YugabyteDB, and TiKV), Stream processors (Flink, Kafka Streams)

Why Materialize does not use RocksDB?

Flink and Kafka Streams use a core (or several) dedicated to each operator

Timely dataflow cores are sharded differently, intentionally to minimize cross-core data movement

Disadvantage: Operators on a worker are cooperatively scheduled, and have to be carefully designed to yield eagerly, or they will block other operators from executing, potentially stalling the entire dataflow graph
Advantage: less cross-core data transfer


feat(deployment): change deployment to known namespace

@deadc I do not expect this to be merged, I thought it would be the easiest way to show you what I changed
See issue #26

This was for debugging purposes.  It does not need to be merged.


Table header resiliency

As an author, I would like the editor / collab to not error out if my table header is malformed so that I can correctly preview my content.
Criteria of acceptance

Editor / Collab / Hast / YJS can handle a table header (td or th) without any text.
Editor / Collab / Hast / YJS can handle a table header (td or th) with two columns.

Additional context
I _think the fix is here.

I was able to reproduce the issue. On this part:

Editor / Collab / Hast / YJS can handle a table header (td or th) with two columns.

the problem happens when the first column in the header is left empty


Resize  windows with mouse or keyboard combination

The problem and the solution
I think the main feature that this wm doesn't have is the possibility to resize windows without any kind of window rule in hypr.conf.
Without this feature i can't use in the way i want my monitor, for exaple: in the same workspace i've got vs code and another application like telegram but don't want telegram taking half of the screen, just a little portion.
like this:

Describe the solution you'd like
I would like to see implemented in this wm an intuitive way to resize windows.
Hope everything is clear

You can resize them with the keyboard. See the splitratio dispatcher.
As for the mouse, it's on my todo list.

pull and recompile, it was added yesterday. (also it's only made for dwindling rn, not master)


error in the power shell when i copy the path of terraform file

Hey chowdamramana
Try not giving spaces to your devops folder

Hey chowdamramana
Try not giving spaces to your devops folder

again i got the same error


Add arrows decoration to line annotation

This PR is adding the optional arrowheads to a line annotation.
TODO

[x] test cases
[x] types
[x] documentation
[x] samples

It's still a working progress. There is a bug that I'm having a look at.

@kurkle before going on, when you have time (no hurry), could you check if there is anything to change or you don't like it?

I set milestone 1.3.0 but feel free to change it.

Remove all redundant options like: xScaleID: 'x' (things that specify the default), as @kurkle already started doing in #631 to make it a bit more readable

It should be ok now

| dist/chartjs-plugin-annotation.esm.js | 12.3 kB | +465 B (+4%) |  |
| dist/chartjs-plugin-annotation.js | 12.4 kB | +469 B (+4%) |  |
| dist/chartjs-plugin-annotation.min.js | 8.39 kB | +391 B (+5%) | 🔍 |

it sounds working well. Thank you


Maximum Pairwise Product In An Array

💥 Proposal
Implement a program in C++ to find the maximum product of two elements of an input array. The approach will be both time and space efficient. This proposal is for NeoAlgo/C-plus-plus.
Have you read the Contributing Guidelines on Pull Requests?
Yes

/assign


Missing wget binary

When running the latest permitio/opal-client:0.7.12 image as docker compose with following command
sh -c "exec ./wait-for.sh http://opal-server:7002/healthcheck --timeout=60 -- ./start.sh"

the ./start.sh script is never executed. The reason is that wget binary does not exists therefor
wget --timeout=1 -q "$HOST" -O /dev/null > /dev/null 2>&1

returns 127 exit code.

/bounty $100

/attempt #655 https://github.com/permitio/opal/pull/662

@ojecborec @gemanor any progress on this issue?

@Abiji-2020 Still waiting for @obsd to help us with failing tests on #662

@Abiji-2020 Still waiting for @obsd to help us with failing tests on #662

okay, incase if you need another hand, i am ready for it .

The log file reads
- Create .env file
- Clone tests policy repo to create test's branch
Cloning into 'opal-tests-policy-repo'...
Load key "/home/runner/work/opal/opal/app-tests/opal-tests-policy-repo-key": error in libcrypto
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.

I've only added wget package to apt-get install statement. This fail seems to be unrelated to my change. Do you have any idea what is going on?

The log file reads
- Create .env file
- Clone tests policy repo to create test's branch
Cloning into 'opal-tests-policy-repo'...
Load key "/home/runner/work/opal/opal/app-tests/opal-tests-policy-repo-key": error in libcrypto
<EMAIL_ADDRESS>Permission denied (publickey).
fatal: Could not read from remote repository.

I've only added wget package to apt-get install statement. This fail seems to be unrelated to my change. Do you have any idea what is going on?

I had checked several PR's in the repo and all of them has this issue, it is not a problem with the code, it is a problem with the docker workflow:
https://github.com/permitio/opal/blob/master/.github/workflows/tests.yml#117
the error is that issue with the SSH key used to clone the repository, which is likely caused by a missing or misconfigured SSH key.

Thank you for the explanation. Is there anything I can help with?

Thank you for the explanation. Is there anything I can help with?

we can ask the maintainers to check on the secrets the correct values are given in the repository. As all the PR currently opened are facing this issue in the docker test.

Hey, we have an issue with the CI at the moment, some expired token of some sort.
I'm waiting for someone else's help, hopefully I will be able to resolve it soon and get this merged 🌟

It is pending 1 reviewer at the moment.

@danyi1212 do we want to merge it?


adds spellCheck, sets it to PLAIN_TEXT by default

It's weird when there are mentions or hashtags and the browser's spell check flags them. We can't disable it for specific text ranges in the input, so this is the best strategy I could come up with:
When there's plain text (no mentions), spell check is enabled. When there's a mention, spell check is disabled. The user can override it with an explicit spellCheck={true} or spellCheck={false}.
Defaulting to the automatic check for mentions existing seems the most intuitive, but I can see arguments against it.

I'm wondering if this is something that's better solved in userland. You can easily keep markup and plain text in a state and set the spellCheck prop based on that:
<MentionsInput spellCheck={this.state.plainText === this.state.markupText}>

If that is a standard DOM property, we should rather simply pass it through, i.e. <MentionsInput spellCheck /> would just work and each user could decide which behavior he/she wants. Trying to outsmart people with implementations that toggle this functionality based on some rather random assumptions almost always leads to weird situations.

Sure, I'll implement it similarly to how it's done in the PR.
const hasMention = this.props.value.indexOf(`{{mention:`) !== -1;
const hasHashtag = this.props.value.indexOf(`{{hashtag:`) !== -1;
<MentionsInput
  spellCheck={!hasMention && !hasHashtag}
  markup="{{__type__:__id__}}"

Using two state keys would be more complex in the case where value changes other than MentionInput's onChange (e.g. loading data from an api).

I think we can fix this by setting this simple css to the mention span
position: relative;
z-index: 1;
background-color: #fff

but for this to work the cursor should jump over the mention not pass through it (else the cursor will be hidden the mentions).


Fix/certicatedup

Removed Duplicate Certificate Loading.
Testing Criteria completed using the same main code, alternating library dependency master vs fork .
MQTT

Cloud device Parameter Passed to IoTaap MQTT Instance.
MQTT Subscription & Unsubscription to topics.
Sending Unique Parameters to MQTT Instance.
Receiving MQTT Payloads from MQTT Instance.
Sending Payloads to devices topic & Events on IoTaap Console.

OTA

Check for auto update forward revisioning to highest version.
Check for auto update backward revisioning.
Check for manual update forward revisioning.
Check for manual update through 20 revisions. Note each runtime.

Notes:
Behaviour consistent when only the lines to set certificate removed.
When either wifiClientSecure.stop(); lines are removed then unhandled exception occurs in a repeatable way.
Thus inability to perform the check update function.
Repeatable behaviour also discovered on hardware reset of connected device to create overflow on the uptime within console.

Upon the advised testing I could find no corresponding issues caused by removing the wifisecureclient.stop in OTA Module, OTA checks and MQTT publishing were available without these lines. I have added a new commit which removes them.
I am sending the testing documentation privately to @igolubic.

After a lot of testing, I can confirm that this solution doesn't work as expected. When automatic update check is enabled and wifiClientSecure.stop(); is not called, the system will fail to check for new updates:
Log:
[2021-03-07 01:21:22] [INFO] - Message successfully published!
[2021-03-07 01:21:22] [INFO] - Checking for updates
sendRequest code=-7
[2021-03-07 01:21:23] [ERROR] - Cannot check for updates
[2021-03-07 01:21:23] [WARNING] - Not connected to MQTT broker
[2021-03-07 01:21:24] [INFO] - Data stored locally!

It will not crash the system, but it will fail to detect the change.
If wifiClientSecure.stop(); is called, then it successfully checks for updates and if needed it starts the update procedure:
Log:
[2021-03-07 01:15:35] [INFO] - Checking for updates
sendRequest code=200
[2021-03-07 01:15:38] [INFO] - No new updates

In both cases, it will fail to switch back to MQTT publishing instantly (and data will be stored locally), it still takes time to reconfigure wifiClientSecure and reconnect to the MQTT and start publishing.
One thing is perfectly clear - it acts really weird, and unstable (not because of @AlexsCode changes, but because of the WiFiClientSecure implementation)

I think that for now we can only remove wifiClientSecure.setCACert( SystemGetCAcertificate());  and move wifiClientSecure.setTimeout(12000 / 1000);  to the mqtt_client.cpp, above wifiClientSecure.setCACert(SystemGetCAcertificate());.
Since update is triggered every 6 hours, just a small percentage of data is stored locally for a short period of time (during update check).
Tested this implementation locally for some time (with OTA_CHECK_TIME set to 120000), and it works fine.

@AlexsCode - can you please implement this, so we can close this PR.

The above changes have been added, and committed with 81c9e934886e5cfdd52ff5fd2c1aa38fb260f097 . Let me know if you require anything more to close the PR @igolubic


support more configs for processor and connection

Changes:

support executionNode for processor
support backPressureDataSizeThreshold and backPressureObjectThreshold for connection

@raymond-ti Nice additions! Could you please confirm that all changes were gofmt'ed.

I used Sublime Text with GoSublime package to edit the code, I think it should be gofmt'ed.


[Decompiler] More fixes for gkernel

Pull Request Test Coverage Report for Build 565003055

97 of 108 (89.81%)  changed or added relevant lines in 7 files are covered.
2 unchanged lines in 2 files lost coverage.
Overall coverage increased (+0.2%) to 68.324%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

decompiler/IR2/FormExpressionAnalysis.cpp
39
42
92.86%

decompiler/util/DecompilerTypeSystem.cpp
0
3
0.0%

decompiler/util/TP_Type.cpp
0
5
0.0%

Files with Coverage Reduction
New Missed Lines
%

decompiler/util/DecompilerTypeSystem.cpp
1
60.61%

decompiler/util/TP_Type.cpp
1
44.07%

Totals

Change from base Build 564550282:
0.2%

Covered Lines:
28847

Relevant Lines:
42221

💛  - Coveralls


Enhance the security of dependencies

Hi,
Considering the security concerns with the dependency github.com/satori/go.uuid, we should replace it.

@jiayouxujin
Does github.com/satori/go.uuid have any security implications? Can you describe it?

@jiayouxujin Does github.com/satori/go.uuid have any security implications? Can you describe it?

@tianshimoyi Sure.
From the results of Docker scan, we can see the following results #18 :
✗ High severity vulnerability found in github.com/satori/go.uuid
  Description: Insecure Randomness
  Info: https://snyk.io/vuln/SNYK-GOLANG-GITHUBCOMSATORIGOUUID-72488
  Introduced through<EMAIL_ADDRESS>  From<EMAIL_ADDRESS>
Then I imported the image to the Snyk platform, and the detailed report is as follows:
Overview:
github.com/satori/go.uuid provides a pure Go implementation of Universally Unique Identifier (UUID).

Affected versions of this package are vulnerable to Insecure Randomness, which produces predictable UUID identifiers due to the limited number of bytes read when using the g.rand.Read function.

Note: This vulnerability was introduced on the master branch of satori/go.uuid after version 1.2.0. This was fixed in commit d91630c8510268e75203009fe7daf2b8e1d60c45.

I checked the commit history of the satori/go.uuid repository, and version 1.2.0 does not include this commit.


[ja] Update page weights under content/ja/docs/concepts/overview/working-with-objects.

This PR updates the page weights under content/ja/docs/concepts/overview/working-with-objects.
This updates the page order.
Related: #37474

/cc @t-inu

@t-inu
I have made an additional update to field-selectors.md  according to #37479.
Could you please review too?
https://github.com/kubernetes/website/pull/38913/files#diff-22776f7de25f2c8876e584de6c5fc56a32880653dbeca40f3164d215d969d61b

@t-inu
Thank you for your review.

Notes regarding files included in the original PR https://github.com/kubernetes/website/pull/37474 and excluded in this PR.
finalizers.md: Exists PR https://github.com/kubernetes/website/pull/38768, and weight is 80 (same as en).
owners-dependents.md: Not applicable because the Japanese page does not exist yet.

The above is one of the advantages of this PR.
If there are not ja pages, the en pages is displayed in the same tree, but they is dispayed in the correct order because the weights are not overlapped.
But in other words, if the weights of the ja page is not adjusted to the weights of the en page, the following situation will occur.

Because the weights of en pages were changed, the order of the tree has changed where ja and en pages are mixed.
The new translated pages will be included the new weight, so the  order will change.

It would be better to change the weight to en as soon as possible.

/assign @nasa9084

/approve
sorry to be late~


Add pull request template

Adds a  pull request template, similar to the ones used  in ody-deployer.
When we create a PR for ods-ci the pull request body will have this content:

[ ] Commits are squashed in a cohesive manner and have meaningful messages (see how)
[ ] Code is formatted following the ODS-CI Robot Framework Style Guide (see how)
[ ] For new automated tests: Automation = Automated in Polarion
[ ] For new automated tests: Automated = Yes in Jira

Signed-off-by: Jorge Garcia Oncins<EMAIL_ADDRESS>

Do not merge until #195 is merged and we have consensus about this

Should we also add a check for
TC is authored in Polarion
TC has corresponding Polarion ID in its Tags
?

[ ] Commits are squashed in a cohesive manner and have meaningful messages (see how)
[ ] Code is formatted following the ODS-CI Robot Framework Style Guide (see how)

For new automated tests:

[ ] Test case is authored in Polarion
[ ]  Automation = Automated in Polarion
[ ] Automated = Yes in Jira
[ ] Test case has tier information and Polarion ID in its Tags (more info here)

New proposal  (point 2 added)

[ ] Commits are squashed in a cohesive manner and have meaningful messages (see how)
[ ] The robocop analyser doesn't detect any issues in the test suites modified by this PR (see check-code-style)
[ ] Code is formatted following the ODS-CI RobotFramework Style Guide

For new automated tests:

[ ] Test case is authored in Polarion
[ ]  Automation = Automated in Polarion
[ ] Automated = Yes in Jira
[ ] Test case has tier information and Polarion ID in its Tags (more info here)

Latest version:

[ ] Commits are squashed in a cohesive manner and have meaningful messages (see how)
[ ] The robocop analyser doesn't detect any issues in the test suites modified by this PR (see check-code-style)
[ ] Code is formatted following the ODS-CI RobotFramework Style Guide
[ ] Jira Link(s):
[ ] An entry has been added to the corresponding RHODS build announcement (see folder)

For new automated tests for RHODS (not ODH):

[ ] Test case is authored in Polarion
[ ]  Automation = Automated in Polarion
[ ] Automated = Yes in Jira
[ ] Test case contains Polarion ID, tier information and all mandatory Tags (more info here)

@jgarciao I'm closing this one, we haven't talked about the need of a template for long time. Feel free to add the topic in the agenda and we can work on this again in the near future


How can I disable the onclick event of button when dragging?

I try like this, but it doesn't work.
$draggables.on( 'dragStart', function( event, pointer, moveVector ) {             $('button').unbind('click');         });         $draggables.on( 'dragEnd', function( event, pointer, moveVector ) {             $('button').bind('click');         });

I have solved it and it is a jquery problem.

How to solve it?


Updating dependabot config to group updates

coverage: 26.816%. remained the same
when pulling ddf8a1d902806a1912aa6ce7a10d612e69a32b65 on feature/dependabot-update
into fc018f280f06e1a19641afb3e03f88ce3d0a2afc on develop.


Sso sessions report / statistics enhancements

Closes https://github.com/Jasig/cas/issues/598
All credit goes to @mindblender

There is still a bit of work to do, but I'll keep the pull open here as we are very close to the final point.

Updated the pull. Should be able to finalize by tomorrow.

Updated docs


Make Invoke-BuildExec/exec display the command.

It would be helpful to make exec display the command it will execute, even if that's controlled with a switch. It would also be useful if the command text (even in the exception handler) were expanded. Here's an implementation illustrating the idea.
function Invoke-BuildExec([Parameter(Mandatory=1)][scriptblock]$Command, [int[]]$ExitCode=0) {
    ${private:*c} = $Command
    ${private:*x} = $ExitCode
    ${private:*cs} = $ExecutionContext.InvokeCommand.ExpandString(($Command.ToString().Split() | Where-Object { $_ }))
    Remove-Variable Command, ExitCode
    Write-Build DarkCyan "CMD: ${*cs}"
    & ${*c}
    if (${*x} -notcontains $global:LastExitCode) {
        *Die "Command {${*cs}} exited with code $global:LastExitCode." 8
    }
}

Same thing but with a switch.
function Invoke-BuildExec([Parameter(Mandatory=1)][scriptblock]$Command, [int[]]$ExitCode=0, [switch]$ShowCommand) {
    ${private:*c} = $Command
    ${private:*x} = $ExitCode
    ${private:*cs} = $ExecutionContext.InvokeCommand.ExpandString(($Command.ToString().Split() | Where-Object { $_ }))
    Remove-Variable Command, ExitCode
    if ($ShowCommand) { Write-Build DarkCyan "CMD: ${*cs}" }
    & ${*c}
    if (${*x} -notcontains $global:LastExitCode) {
        *Die "Command {${*cs}} exited with code $global:LastExitCode." 8
    }
}

I prefer without a switch, but I don't know if you want to have the default behavior to show the command.

@wekempf It looks like a reasonable idea, thank you. It should be a switch (sorry about your preferred default), some simpler name like -Echo may be used. Or maybe an IB option or a script level option, this needs some thinking.

Both a switch and a more global option maybe? I can see wanting to show it selectively as well as just changing the behavior for a whole script.

I'm already doing this in some of my scripts by simply overwriting the function with my own. That might be a worthwhile example to add (not everyone will be aware that you can redefine anything in PowerShell).

With the switch, the script level option will be kind of available. Modern PowerShell has a way to specify the default for a command parameter, IIRC.

I'd vote for just the switch then.

I like this, maybe just think about adding a note in the cmdlet help that people should use -Echo (or whatever) with care, b/c it could easily lead to leaking sensitive information/credentials...

@mwallner  Good point. Wonder if there's a way to specify variables not to expand here. Definitely a reason to put this behind a switch, though.

Oh... I missed the whole variable expansion in the code, I thought about just the script block printing. I am not sure I like this, yet. This needs even more thinking, does not look safe...

The variable expansion is quite important in my scenarios, as figuring out what the build did on failures is crucial. I understand the security concerns, however, so something like an -ExcludeExpansion list maybe?

so something like an -ExcludeExpansion list maybe

Or rather -EchoExpandVariables, to be secure by default.
I wonder if this expansion works in older PS versions, too.

Side question. Why .Split() | Where-Object { $_ }? I.e. why not to expand on the whole string?

I'll be honest, that was borrowed code. I think the split is an odd way to deal with spacing concerns, rather than just trimming the string, but that's purely a guess on my part.
Selectively expanding will be interesting, given variables can be represented multiple ways. With and without scopes, with and without curly braces, some other way I'm not familiar with?

How about this approach which avoids and solves issues
	Get-Variable x, y | Format-Table
	exec {something -with $x $y} -echo

I do not think we should use ExpandString, it is supposed to invoke code, too, not just expand variables:
function boom {
	throw 'boom'
}

$text = '$(boom)'
$ExecutionContext.InvokeCommand.ExpandString($text)

As a result, I am thinking of just this simple change:
function Invoke-BuildExec([Parameter(Mandatory=1)][scriptblock]$Command, [int[]]$ExitCode=0, [switch]$Echo) {
	if ($Echo) {
		Write-Build 3 "exec {$Command}"
	}
	${private:*c} = $Command
	${private:*x} = $ExitCode
	Remove-Variable Command, ExitCode, Echo
	& ${*c}
	if (${*x} -notcontains $global:LastExitCode) {
		*Die "Command {${*c}} exited with code $global:LastExitCode." 8
	}
}

Is it still needed in this form without string expansion? I do not want to engage string expansion for safety reasons.
By the way, how to enable -Echo for the whole script or task (depends on the place):
$PSDefaultParameterValues=@{'Invoke-BuildExec:echo'=$true}

So, do we proceed with this simple change or it's not worth it?

OK, some development, see the created branch and maybe try (use the new Show-ExecEcho.ps1 and updated Invoke-Build.ps1). Unlike ExpandString which may accidentally (or worse) invoke code, Show-ExecEcho.ps1 only evaluates variables. This is safer. But the code is more complex, so I decided to use a separate helper script.
Example This script
$foo = 'foo'
$env:SOME_VAR = 'SOME_VAR'

task t1 {
	$bar = 'bar'

	exec -echo {  cmd /c echo $script:foo $env:SOME_VAR  }

	exec -echo {
		# bar
		cmd /c echo $bar $env:SOME_VAR
	}
}

outputs

After playing with this I started to like the idea with this implementation. Also, after some tweaks the code is simple enough for the main script, the helper script is not needed.

Done in v5.7.0


Additional configurations on Storage Bucket

** Which Category is your question related to? **
Storage
** What AWS Services are you utilizing? **
Amplify-CLI, Storage, Auth
** Provide additional details e.g. code snippets **
Somehow related to #2279, on being able to extend cloudformation templates generated by Amplify.
My app just uses Storage and Auth (no API) to allow users store and retrieve files.
I create a storage bucket with amplify add storage. So far so good.
I now want to tweak some of this bucket settings, mainly:

Enable transfer acceleration
Enable lifecycles
Add custom tags

I could modify amplify/backend/storage/stackName/s3-cloudformation-template.json, but my changes will be overwritten by any amplify storage update.
Is there a way to add this via an additional "plugin-like" template file? Or do I need to manage my  bucket without amplify as a "custom resource", in which case I loose the role policies managed by Amplify.

@DanielMuller You could go ahead and modify the Cloudformation file for the S3 storage resource localted in amplify/backend/storage/<resource-name> directory to add these additional configurations if you need it.

Also, I don't believe amplify storage update will overwrite your changes. Did you verify this behavior?

Also, I don't believe amplify storage update will overwrite your changes. Did you verify this behavior?

Just tested it with "Acceleration" and a "Lifecycle", and indeed it doesn't get overwritten after amplify storage update (I changed the auth rules and added a trigger).
I wrongly assumed that the Cloudformation template would be rewritten on any CLI change without even trying it out.


In-source documentation

Summary
#310
Details and comments
Trying to mimic https://github.com/Qiskit/qiskit-terra/pull/3325 and https://github.com/Qiskit/qiskit-terra/pull/3396

This PR will:

Update the infrastructure for in source documentation in Ignis
Update in source documentation of coherence and measurement mitigation: add Jupyter examples and update init files

Afterwards the rest of the modules will be similarly updated by their developers.

Qiskit/qiskit-terra#3396 is in fact not relevant

The coherence part is ready, so I’m removing the WIP prefix in the title. Next week in this PR or a separate one I’ll handle measurement mitigation.

@mtreinish Could you please take a look at the failing checks

It's failing the lint, can you fix @yaelbh ?

Fixed, sorry for not noticing myself

There Azure doc failure looks to be due to issues with the jupyter-execute blocks. It is saying that it doesnt recognize these directives. @nonhermitian is there something extra that needs to be done to get these to work?
In the mean-time we could get this passing and merged by changing all the jupyter blocks to simple code blocks that aren't executed.:
.. code-block:: python

    # Py code example...

You need jupyter-sphinx from pip

Also need to have jupyter_sphinx.execute in extensions section of docs config

Closing since I think this was superseded by all the individual doc PRS


Update Composer dependencies (2022-02-05-00-12)

Visual regression test passed!
View the visual regression test report


Use appsDomain if it is set when determining Route host

See this doc: https://docs.openshift.com/container-platform/4.12/networking/ingress-operator.html#nw-ingress-configuring-application-domain_configuring-ingress
If appsDomain is set, it should override whatever is set in domain when creating a Route.
This illustrates the problem:
[loganmc10@fedoralogan ~]$ oc get route -A
NAMESPACE                             NAME                       HOST/PORT                                                             PATH        SERVICES                   PORT    TERMINATION            WILDCARD
open-cluster-management-agent-addon   klusterlet-addon-workmgr   workmgr-f237a30cbe328fa345042ce3db57ffb5.apps.smb.bacoosta.com                    klusterlet-addon-workmgr   <all>   passthrough            None
openshift-authentication              oauth-openshift            oauth-openshift.apps.smb2.bacoosta.com                                            oauth-openshift            6443    passthrough/Redirect   None
openshift-ingress-canary              canary                     canary-openshift-ingress-canary.apps.smb2.bacoosta.com                            ingress-canary             8080    edge/Redirect          None
openshift-monitoring                  alertmanager-main          alertmanager-main-openshift-monitoring.apps.smb2.bacoosta.com         /api        alertmanager-main          web     reencrypt/Redirect     None
openshift-monitoring                  prometheus-k8s             prometheus-k8s-openshift-monitoring.apps.smb2.bacoosta.com            /api        prometheus-k8s             web     reencrypt/Redirect     None
openshift-monitoring                  prometheus-k8s-federate    prometheus-k8s-federate-openshift-monitoring.apps.smb2.bacoosta.com   /federate   prometheus-k8s             web     reencrypt/Redirect     None
openshift-monitoring                  thanos-querier             thanos-querier-openshift-monitoring.apps.smb2.bacoosta.com            /api        thanos-querier             web     reencrypt/Redirect     None

If I change my cluster Ingress domain to smb2.bacoosta.com by changing the appsDomain value, every Route uses that new domain, except the klusterlet-addon-workmgr route, which continues to use the old smb.bacoosta.com domain

This seems to be a regression, possibly starting from here: https://github.com/stolostron/multicloud-operators-foundation/pull/556 (affecting backplane 2.3 and 2.4 currently)


Document ordering of GTOs

Hi, and very nice project!
I'm struggling a bit interfacing with an external tool because I need to know the order in which the GTOs are constructed from the basis. Eg will a p gaussian give (m=-1, m=0, m=1), or (px, py, pz), or some other ordering? I was not able to find this in the documentation or reverse engineer it from the matrices. Apologies if I missed somewhere obvious.

Just print mol.ao_labels(), the order is very clear.

Thanks! I actually got around to it through https://github.com/sunqm/libcint/blob/master/doc/program_ref.pdf, which appears to match.
I had a bit of trouble figuring out the pattern, comparing to https://en.wikipedia.org/wiki/Table_of_spherical_harmonics#Real_spherical_harmonics. Is it right that they match, except for the fact that the l=1 are relabellings of the usual ones? I also had a bit of trouble because of the fact that the m=0 for l>=2 appear to differ, but if I understand correctly this is just convention and what pyscf calls "z^2" is actually "2z^2-x^2-y^2"?

Yeah the libcint manual is helpful. I believe the ordering for l=1 is (1,-1,0) rather than the standard CCA ordering, which would be (-1,0,1).
I am not sure the answer to your "z^2" question.

Yes, that's what I see as well for the l=1 case. I believe the z^2 is indeed (very confusing, to me at least) "shorthand" for 2z^2-x^2-y^2: if it was not, then that orbital would have a non-zero overlap with the s, which is not the case.

Except l=1, other spherical GTOs follow the convention of
https://en.wikipedia.org/wiki/Table_of_spherical_harmonics#Real_spherical_harmonics
dz^2 is the orbital of l=2, m=-2. The overlap between d_{-2} and s_0 should be
zero regardless of convention. You may find error of ~1e-18. They are numerical
round off error.

dz^2 is the orbital of l=2, m=-2

You mean m=0, right? Yes I agree, my confusion was that "dz^2" does not mean z^2/r^2 but rather (2z^2-x^2-y^2)/r^2. I suppose it's "standard", but still confusing.

You are correct. dz^2 is m=0, short for (2z^2-x^2-y^2)/r^2

Thank you, that's very helpful! It could be nice to document this somewhere - I saw a dev docs is planned, that could be a nice place for it. I'm closing this issue.


validate data sent to create/update for json-api compliance

I'm :+1: to the verify step, but I feel like sanitization belongs in the source.
I think our source methods (create/update/delete) need to be smart enough to receive a json-api compliant object and do all the bookshelf methods need to deal with relations, etc. Thoughts?

Also, this sanitization method will break down for sending compound documents.

Putting the sanitization it in our source facade makes sense.
Under the base spec, only single data objects are allowed on a create or update. Supporting extensions is going to have to override the base spec in a lot more than just this method. (e.g. all of the content-type and accept header verification will have to allow for and check for specific extensions on request and response).
I guess I'm taking the TDD view and I'm willing to punt on it until we move to fully support the bulk extension.

Moved sanitization to source.
I was tempted to just automatically delete any props that weren't on the source model, but it should be explicitly aware of what we're adding because of json-api, and error out when the user sends a prop that isn't on the model and not required by json-api.

:+1: to erroring out on invalid props rather than silently removing them.
I was speaking with @bmac about this today. I've done a bunch of research into schema validation and I think incorporating json-schema definitions on the sources actually makes a ton of sense. This is basically a reversal of punting on route validation in endpoints issue #11.
This would gives us the "holy grail" of client-side validation using the same configuration as the server (and enhance our ability to self document / automate testing). Joi can't be used on the client without browserify, and the resulting file is pretty massive. It's also not based on a spec.
I'm pretty sure I want to use is-my-json-valid.
ref: http://gilesbowkett.blogspot.com/2015/01/why-panda-strike-wrote-fastest-json.html
ref: https://www.pandastrike.com/posts/20150123-congrats-imjv

Shoot, can you rebase this against master?

rebased.

:+1:


npm install is giving me 2.1.4

Related to closed #310
When I use npm install ng-flow or with yarn, I'm getting version 2.1.4 and not the last one (2.7.4)

@mgohin #310 actually resolved your problem - use https://www.npmjs.com/package/@flowjs/ng-flow

No it didn't :D

I managed to install v2.7.4 with the following: npm install @flowjs/ng-flow.
https://www.npmjs.com/package/@flowjs/ng-flow

Sorry, missread your answer ... :(
Thx 👍


Custom punctuation

Analog to custom_keyword!, custom_punctuation! allows a user of the library to treat a sequence symbols as one token.
Closes #502.
Original contents
Only loosely tested (to compile) so far, and using CustomPunctuation the same as CustomKeyword results in conflicting implementations of both token::Sealed and token::Token, as the two traits could both be implemented on the same type. For demonstrative purposes, CustomKeyword has been commented out.
The approach that will work for both traits will probably be making token::Token a supertrait and exposing token::Sealed in $crate::export to allow our macros to implement it.

CustomKeyword is private, you can rename it and use the same trait for both custom keywords and custom punctuation. That way only one blanket impl is necessary.
Here is a macro to compute length of a custom punctuation without str::len:
macro_rules! len {
    (<=) => { 2 };
    (>) => { 1 };
    /* ... */
    ($first:tt $($rest:tt)+) => {
        len!($first) $(+ len!($rest))*
    };
}

fn main() {
    let spans = [0; len!(<=>)];
    println!("{}", spans.len());
}

I think the macro will still have to take punctuation in a string, though, as stringify!(<=>) is "<= >". Or I could modify the punct helpers to skip over whitespace in the string if the ergonomy of the custom_punctuation! is more important.
The other idea I had was to have a struct CustomPunctuation(&'static [char]) instead of using a trait.
(Given that this macro is only really useful in functionlike proc_macro town, we could write it as a functionlike proc_macro and only expose it on Rust >= 1.29..... but that sounds like a bad idea and way too overkill.)

I've not access to my development machine at this exact moment so I'm dropping this new sketch here:

trait CustomToken<I: CustomIdent, P: CustomPunct> {
    fn token_str() -> &'static str;
    fn display() -> &'static str;
    fn spans(&self) -> &[Span]
}

// Keyword as before
impl CustomToken for Keyword {
    fn token_str() -> &'static str { stringify!($ident) }
    fn display() -> &'static str { concat!("`", stringify!($ident), "`") }
    fn spans(&self) -> &[Span] { &[self.span] }
}

macro custom_punctuation!( Punct($symbol:tts) ) {

struct Punct {
    pub spans: [Span; len!($symbol)];
}

impl CustomToken for Punct {
    fn token_str() -> &'static str { stringify!($symbol) } // WRONG
    fn display() -> &'static str { concat!("`", stringify!($symbol), "`") } // WRONG
    fn spans(&self) -> &[Span] { &self.spans }
}
impl Parse for Punct {
    fn parse(input) -> Result<Self> {
        // must expose parsing::punct somehow (or duplicate)
        Ok(Token { spans: parsing::punct(input, Self::token_str())? })
    }
}
impl ToTokens for Punct {
    fn to_tokens(&self, &mut tokens) {
        // must expose printing::punct somehow (or duplicate)
        printing::punct(Self::token_str(), self.spans(), tokens)
    }
}

}

Side thought: might it make sense to robust-ify these macros a bit more and use them internally as well? Or at the very least some of the internal bits should be reusable between tokens that do and don't go through CustomToken.
macro custom_keyword!($meta $kw) {
    impl_custom_token!($meta $kw);
    impl_token_parse!($kw);
    // etc
}

macro internal_keyword!($meta $kw) {
    impl_internal_token!($meta $kw);
    impl_token_parse!($kw);
    // etc
}

(Just using macro keyword for compactness)

Here's a stringify without spaces:
macro_rules! stringify_punct {
    ($($tt:tt)*) => {
        concat!($(stringify!($tt)),*)
    };
}

fn main() {
    let s: &'static str = stringify_punct!(<=>);
    println!("{:?}", s);
}

I'm not sure exactly where things should end up; I just chose arbitrarily to group stuff as it is in the above commit. test_rustc_precedence is failing on my end (%1 is not a valid Win32 application), so I'm letting Travis test.

Alright, this is working (modulo the use of pub(restricted) syntax which breaks the 1.15 build). I rebased down to one commit since it's one step and most of the commits were just "try fixes".
I just need to move stuff to the correct location, which will probably be just to not use $crate::custom_token::punct but rather #[doc(hidden)] pub the helpers in $crate::token, and use those directly. I'm also open to inlining the helpers manually as is (still) done for custom_keyword); which would you prefer @dtolnay?

That should address everything!

Still working on those two fixes?

Over in https://github.com/rust-lang/gll/pull/77 I discovered that the inbuilt proc_macro doesn't properly maintain joint-ness of Punct, so this is probably dead in the water until that's fixed upstream. (It still works with proc_macro2, just not proc_macro. Exposing this and having behavior differ depending on backend seems like the worst option, sadly.)

It works fine unless you've hit https://github.com/rust-lang/rust/issues/43081. For example try writing m!{%%} in dev/main.rs and it parses as:
Macro {
    path: Path {
        leading_colon: None,
        segments: [
            PathSegment {
                ident: Ident {
                    ident: "m",
                    span: #0 bytes(94..95),
                },
                arguments: None,
            },
        ],
    },
    bang_token: Bang,
    delimiter: Brace(
        Brace,
    ),
    tts: TokenStream [
        Punct {
            ch: '%',
            spacing: Joint,
            span: #0 bytes(97..98),
        },
        Punct {
            ch: '%',
            spacing: Alone,
            span: #0 bytes(98..99),
        },
    ],
}

You're right, there's still a roundtrip through strings in the library >.<


Correct the positive and negative persona

In the make_dataset, the positive and negative persona looks like it's reverse.

Thanks for this--I'll need to take a deeper look at the tests, since I'm not sure why they're passing with the flipped vector (it's possible I reduced the training examples too much in my attempt to speed up the testsuite, and the vector isn't being trained right...)

Fixed by #34


Linux export/import window doest not show properly

Describe the bug
Trying to export endpoints, the window does not show any character, everything as squares .
To Reproduce
1 . Go to Export / Import
Screenshots

Desktop (please complete the following information):

OS: Linux Manjaro
Installation Method: snap
App Version: insomnia 2020.4.1 from Insomnia (gschier1990)

This looks very much related to https://github.com/Kong/insomnia/issues/2691 🤔

yes, it's same issue. . I will close this one then. Thanks!

No worries, thank you for reporting! From the activity in the other issue we're still trying to get to the bottom of it. Even though this issue is closed it will be linked in #2614 when being investigated. 😄


[WIP] Refactor activate / sync survey process into use cases

Towards #1429.

?G

/gcbrun

/gcbrun

@shobhitagarwal1612 RFAL! Thank you!

/gcbrun

/gcbrun


Add support to use custom chat template for usage with koboldcpp

Is your feature request related to a problem? Please describe.
Koboldcpp uses Alpaca as the default chat template for all v1 api completion requests, causing models with other chat templates to not work correctly.
Describe the solution you'd like
Koboldcpps work around is described here: https://github.com/LostRuins/koboldcpp/pull/466 and https://github.com/LostRuins/koboldcpp/pull/466/files/fb6b9a8c41e8e8000fb18e119be3704c646ba55e.
It would be great if we could specify a custom chat template for a model to use with koboldcpp.
Describe alternatives you've considered
I currently hardcoded the chat template for Llama-3 into backend/apps/openai/main.py: async def generate_chat_completion() adding this code:
payload["adapter"] = {"system_start":"<|start_header_id|>system<|end_header_id|>\n\n", "system_end":"<|eot_id|>", "user_start":"<|start_header_id|>user<|end_header_id|>\n\n", "user_end":"<|eot_id|>", "assistant_start":"<|start_header_id|>assistant<|end_header_id|>\n\n", "assistant_end":"<|eot_id|>"}

Fyi, the adapter can be sent over the API, but it also accepts plain JSON files that can be loaded independently of any frontend such as open-webui.
Here's a sample file: https://github.com/LostRuins/koboldcpp/wiki#what-is---chatcompletionsadapter
When launching KoboldCpp, you can apply this adapter manually using this technique. It can also be selected in the GUI loader.

Or if using CLI, with the --chatcompletionsadapter launch flag.
Of course, this does not preclude sending an adapter over the API, and if an adapter is sent over the API it will take precedence over the one selected on load.

Sorry if out of topic of issue but are openwebui support koboldcpp and horde api?

Koboldcpp provides a OpenAI compatible endpoint for completions and chat completions, so it should.

On a side note, I've noticed many gguf's have a jinja2 based template value in them, that at least llama-cpp-python uses. Does koboldcpp support or plan to support that? If so, that could be one workaround for this problem.


Do not log stack trace of Http(Status|Response)Exception

Motivation:
HttpResponseException and HttpStatusException are raised by user code so we do not have to log the stack trace.
Modification:

Do not log the stacktrace of HttpStatusException and HttpResponseException in LoggingService.

Result:

Close #3053
You no longer see the stacktrace of HttpStatusException and HttpResponseException in logging.

/cc @eonezhang

Thanks for reviewing. 😉


import users: prefix isn't being used

I set the prefix to weird- before clicking import users but it continued to use lausd-.

Did these users pre-exist?

Mmm let me try

Fixed


option to turn off lateral flow for the soil models

Purpose
This adds the option to turn off the flow of heat and water (soil energy/hydrology) or of water (richards equation) even in the case of a 3d domain (hybrid box or spherical shell).
Closes #234
Content

Adds an additional field to the soil models (EnergyHydrology and RichardsModel) with a default value of true (when using the outer constructor).
Adds additional argument to horizontal_components! function
Adds a new method for that function
In summary, we now also use multiple dispatch on the value of a boolean flag in the
horizontal_components! function. This function now dispatches off of the type of the domain, value of the lateral_flow flag, and the model type as follows:

domain::Column -> no horizontal components, regardless of the lateral_flow flag value
domain::Union{HybridBox,SphericalShell}, lateral_flow::Val{false} -> no horizontal components, regardless of soil model type.
domain::Union{HybridBox,SphericalShell}, lateral_flow::Val{true}, model::RichardsModel -> adds in horizontal flow of water
domain::Union{HybridBox,SphericalShell}, lateral_flow::Val{true}, model::EnergyHydrology -> adds in horizontal flow of water and heat.

Review checklist
I have:

followed the codebase contribution guide: https://clima.github.io/ClimateMachine.jl/latest/Contributing/
followed the style guide: https://clima.github.io/ClimateMachine.jl/latest/DevDocs/CodeStyle/
followed the documentation policy: https://github.com/CliMA/policies/wiki/Documentation-Policy
checked that this PR does not duplicate an open PR.

In the Content, I have included

relevant unit tests, and integration tests,
appropriate docstrings on all functions, structs, and modules, and included relevant documentation.

[X] I have read and checked the items on the review checklist.

bors r+

bors r+


use fiber version with cross-compilation bug fixed

Had an issue when trying to build release binaries. I get this error:
task: [build-osx] GOOS=darwin GOARCH=amd64 CGO_ENABLED=0 go build -ldflags="-w -s -X main.appVersion=0.5.0"  -o build/darwin-amd64/valkyrie
# github.com/gofiber/fiber/v2/internal/gopsutil/mem
../../go/pkg/mod/github.com/gofiber/fiber/v2@v2.41.0/internal/gopsutil/mem/mem_darwin_nocgo.go:21:14: undefined: invoke
task: Failed to run task "build-osx": exit status 2

Apparently its fixed in https://github.com/gofiber/fiber/issues/2293
Tested locally and confirmed its working.

Added task to remind us to upgrade to fiber 2.42.0 once available.


Resolves #66

Resolves #66 - Simply modified the existing LengthFilter to also filter out bad Aliases.
As much as we're all Regex wizards, it's easier to simply filter out bad results after they've been caught than to try and wedge a {,64} into a new matching group.
Used the existing length filter where emails >256 were being filtered out

Love it! All for killing unnecessary regex complexity, the only hesitation I have is that where there are other places in the system that need to apply the same logic, there needs to be additional code and not just a regex update. But that's less important than actually making stuff work now 🙂

What places in the system do you mean could also use the same logic?
The current design is to use one all-encompassing loose-on-strictness Regex query, and then filtering out the bad results using filters (possibly including additional more strict sub-Regex queries)
The more strict (or complex) the main Regex, the more cycles that are run on the raw content which brings old performance issues back up

Not that I'm not all for encouraging new contributors to try and take a crack at things
The filter system also allow new contributors to try out entirely new Regex queries for catching emails from raw input without breaking hardline rules

Other places include:

When signing up for individual notifications
When signing up for domain notifications
When searching for a specific email address

I'm not overly precious about it needing to be regex though, I just need to maintain the logic across other parts of the system that aren't already OSS.


CLI tooling tracking issue

Original merge from PR #47. References #43

[x] Easy to read error message if client doesn't connect - comment
Something like

Failed to connect to driver; please ensure driver is running and working. Driver must also not be in use by any other tools, e.g. gui apps

[x] Make RequestState more global - comment
[x] Use smallest available IDs - comment
[x] For add/add-mode/remove-mode, allow specifying custom refresh rates per monitor mode. Making it clearer to type is also a bonus. - comment
[x] Deprecate mode indexes and allow selection of monitors by unique WIDTH/HEIGHT. Potentially also allow deleting only refresh rate instead of entire mode. - comment. UX TBD
[x] Make client connection global / always store requested state so it can be accessible anywhere. - comment
[x] With global state, other checks can be made smarter, such as displaying an error when user tries to remove a monitor ID that doesn't exist.

Cc @kylewlacy

For anyone who wants to try it today, please see this discussion:
https://github.com/MolotovCherry/virtual-display-rs/discussions/60

Finished in #97


Converted Regular Expression to ES5

Generated an equivalent regex using regexpu so that it does not break in IE.
This is to fix https://github.com/fb55/entities/issues/209

@manish-kothari When can we push this patch version? It'd help a lot as a lot of people are facing backward compat issues.

@shobhitchittora I have raised a pull request, https://github.com/fb55/entities/pull/212 it is yet to be approved and merged. It caused a lot of issues for us as well, we worked around this by forking and using this forked version.

Yeah! For us it's too deep down the dependency tree to use a fork.