bad coregistrations

Describe the bug
XCP-CPAC_compare.pdf
subjects with bad coregistrations
HRC
20415
NKI
A00033752, A00052461, A00053171, A00057372, A00060846, A00062361, A00066237, A00066580, A00076594, A00077412, A00079724
To reproduce
No response
Preconfig

[ ] default
[ ] abcd-options
[ ] anat-only
[ ] blank
[ ] ccs-options
[ ] fmriprep-options
[ ] fx-options
[ ] monkey
[ ] monkey-ABCD
[ ] ndmg
[ ] nhp-macaque
[ ] preproc
[X] rbc-options
[ ] rodent

Custom pipeline configuration
No response
Run command
No response
Expected behavior

bad coregistrations retry or fail

Acceptance criteria

[ ] bad coregistrations retry or fail

Screenshots
No response
C-PAC version
v1.8.4, v1.8.5-dev
Container platform
Singularity
Docker and/or Singularity version(s)
Singularity 3.8.5-2.el7
Additional context
:link: workng doc

runseednative spacetemplate space
sub-A00062361_ses-BAS177742777

Three subjects with this registration configuration
registration_workflows:

  anatomical_registration:

    run: On

    # The resolution to which anatomical images should be transformed during registration.
    # This is the resolution at which processed anatomical files will be output.
    resolution_for_anat: 1mm

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_brain_template: /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain.nii.gz

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_template: /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}.nii.gz

    # Template to be used during registration.
    # It is not necessary to change this path unless you intend to use a non-standard template.
    T1w_brain_template_mask: /usr/share/fsl/5.0/data/standard/MNI152_T1_${resolution_for_anat}_brain_mask.nii.gz

    # Register skull-on anatomical image to a template.
    reg_with_skull: Off

    registration:

      # using: ['ANTS', 'FSL', 'FSL-linear']
      # this is a fork point
      #   selecting both ['ANTS', 'FSL'] will run both and fork the pipeline
      using: [ANTS]

      # option parameters
      ANTs:

        # If a lesion mask is available for a T1w image, use it to improve the ANTs' registration
        # ANTS registration only.
        use_lesion_mask: Off

        # ANTs parameters for T1-template-based registration
        T1_registration:
          - collapse-output-transforms: 1
          - dimensionality: 3
          - initial-moving-transform:
              initializationFeature: 0
          - transforms:
            - Rigid:
                convergence:
                  convergenceThreshold: 1e-06
                  convergenceWindowSize: 20
                  iteration: 100x100
                gradientStep: 0.05
                metric:
                  metricWeight: 1
                  numberOfBins: 32
                  samplingPercentage: 0.25
                  samplingStrategy: Regular
                  type: MI
                shrink-factors: 2x1
                smoothing-sigmas: 2.0x1.0vox
                use-histogram-matching: On
            - Affine:
                convergence:
                  convergenceThreshold: 1e-06
                  convergenceWindowSize: 20
                  iteration: 100x100
                gradientStep: 0.08
                metric:
                  metricWeight: 1
                  numberOfBins: 32
                  samplingPercentage: 0.25
                  samplingStrategy: Regular
                  type: MI
                shrink-factors: 2x1
                smoothing-sigmas: 1.0x0.0vox
                use-histogram-matching: On
            - SyN:
                convergence:
                  convergenceThreshold: 1e-06
                  convergenceWindowSize: 10
                  iteration: 100x70x50x20
                gradientStep: 0.1
                metric:
                  metricWeight: 1
                  radius: 4
                  type: CC
                shrink-factors: 8x4x2x1
                smoothing-sigmas: 3.0x2.0x1.0x0.0vox
                totalFieldVarianceInVoxelSpace: 0.0
                updateFieldVarianceInVoxelSpace: 3.0
                use-histogram-matching: On
                winsorize-image-intensities:
                  lowerQuantile: 0.005
                  upperQuantile: 0.995

        # Interpolation method for writing out transformed anatomical images.
        # Possible values: Linear, BSpline, LanczosWindowedSinc
        interpolation: LanczosWindowedSinc

      FSL-FNIRT:

        # Configuration file to be used by FSL to set FNIRT parameters.
        # It is not necessary to change this path unless you intend to use custom FNIRT parameters or a non-standard template.
        fnirt_config: T1_2_MNI152_2mm

        # The resolution to which anatomical images should be transformed during registration.
        # This is the resolution at which processed anatomical files will be output.
        # specifically for monkey pipeline
        ref_resolution: 2mm

        # Reference mask for FSL registration.
        ref_mask:

        # Template to be used during registration.
        # It is for monkey pipeline specifically.
        FNIRT_T1w_brain_template:

        # Template to be used during registration.
        # It is for monkey pipeline specifically.
        FNIRT_T1w_template:

        # Interpolation method for writing out transformed anatomical images.
        # Possible values: trilinear, sinc, spline
        interpolation: sinc

        # Identity matrix used during FSL-based resampling of anatomical-space data throughout the pipeline.
        # It is not necessary to change this path unless you intend to use a different template.
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat

        # Reference mask with 2mm resolution to be used during FNIRT-based brain extraction in ABCD-options pipeline.
        ref_mask_res-2:

        # Template with 2mm resolution to be used during FNIRT-based brain extraction in ABCD-options pipeline.
        T1w_template_res-2:

    overwrite_transform:

      run: Off

      # Choose the tool to overwrite transform, currently only support 'FSL' to overwrite 'ANTs' transforms in ABCD-options pipeline.
      # using: 'FSL'
      using: FSL

  functional_registration:

    coregistration:

      # functional (BOLD/EPI) registration to anatomical (structural/T1)

      run: On

      # reference: 'brain' or 'restore-brain'
      # In ABCD-options pipeline, 'restore-brain' is used as coregistration reference
      reference: brain

      # Choose FSL or ABCD as coregistration method
      using: FSL

      # Choose brain or whole-head as coregistration input
      input: brain

      # Choose coregistration interpolation
      interpolation: trilinear

      # Choose coregistration cost function
      cost: corratio

      # Choose coregistration degree of freedom
      dof: 6

      # Extra arguments for FSL flirt
      arguments:

      func_input_prep:

        # Choose whether to use functional brain or skull as the input to functional-to-anatomical registration
        reg_with_skull: Off

        # Choose whether to use the mean of the functional/EPI as the input to functional-to-anatomical registration or one of the volumes from the functional 4D timeseries that you choose.
        # input: ['Mean_Functional', 'Selected_Functional_Volume', 'fmriprep_reference']
        input: [fmriprep_reference]

        Mean Functional:

          # Run ANTs’ N4 Bias Field Correction on the input BOLD (EPI)
          # this can increase tissue contrast which may improve registration quality in some data
          n4_correct_func: Off

        Selected Functional Volume:

          # Only for when 'Use as Functional-to-Anatomical Registration Input' is set to 'Selected Functional Volume'.
          #Input the index of which volume from the functional 4D timeseries input file you wish to use as the input for functional-to-anatomical registration.
          func_reg_input_volume: 0

      boundary_based_registration:

        # this is a fork point
        #   run: [On, Off] - this will run both and fork the pipeline
        run: [On]

        # Standard FSL 5.0 Scheduler used for Boundary Based Registration.
        # It is not necessary to change this path unless you intend to use non-standard MNI registration.
        bbr_schedule: /usr/share/fsl/5.0/etc/flirtsch/bbr.sch

        # reference for boundary based registration
        # options: 'whole-head' or 'brain'
        reference: brain

        # choose which FAST map to generate BBR WM mask
        # options: 'probability_map', 'partial_volume_map'
        bbr_wm_map: partial_volume_map

        # optional FAST arguments to generate BBR WM mask
        bbr_wm_mask_args: -bin

    EPI_registration:

      # directly register the mean functional to an EPI template
      #   instead of applying the anatomical T1-to-template transform to the functional data that has been
      #   coregistered to anatomical/T1 space
      run: Off

      # using: ['ANTS', 'FSL', 'FSL-linear']
      # this is a fork point
      # ex. selecting both ['ANTS', 'FSL'] will run both and fork the pipeline
      using: [ANTS]

      # EPI template for direct functional-to-template registration
      # (bypassing coregistration and the anatomical-to-template transforms)
      EPI_template: s3://fcp-indi/resources/cpac/resources/epi_hbn.nii.gz

      # EPI template mask.
      EPI_template_mask:

      ANTs:

        # EPI registration configuration - synonymous with T1_registration
        # parameters under anatomical registration above
        parameters:

        # Interpolation method for writing out transformed EPI images.
        # Possible values: Linear, BSpline, LanczosWindowedSinc
        interpolation: LanczosWindowedSinc

      FSL-FNIRT:

        # Configuration file to be used by FSL to set FNIRT parameters.
        # It is not necessary to change this path unless you intend to use custom FNIRT parameters or a non-standard template.
        fnirt_config: T1_2_MNI152_2mm

        # Interpolation method for writing out transformed EPI images.
        # Possible values: trilinear, sinc, spline
        interpolation: sinc

        # Identity matrix used during FSL-based resampling of BOLD-space data throughout the pipeline.
        # It is not necessary to change this path unless you intend to use a different template.
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat

    func_registration_to_template:

      # these options modify the application (to the functional data), not the calculation, of the
      # T1-to-template and EPI-to-template transforms calculated earlier during registration

      # apply the functional-to-template (T1 template) registration transform to the functional data
      run: On

      # apply the functional-to-template (EPI template) registration transform to the functional data
      run_EPI: Off

      output_resolution:

        # The resolution (in mm) to which the preprocessed, registered functional timeseries outputs are written into.
        # NOTE:
        #   selecting a 1 mm or 2 mm resolution might substantially increase your RAM needs- these resolutions should be selected with caution.
        #   for most cases, 3 mm or 4 mm resolutions are suggested.
        # NOTE:
        #   this also includes the single-volume 3D preprocessed functional data,
        #   such as the mean functional (mean EPI) in template space
        func_preproc_outputs: 2mm

        # The resolution (in mm) to which the registered derivative outputs are written into.
        # NOTE:
        #   this is for the single-volume functional-space outputs (i.e. derivatives)
        #   thus, a higher resolution may not result in a large increase in RAM needs as above
        func_derivative_outputs: 2mm

      target_template:

        # choose which template space to transform derivatives towards
        # using: ['T1_template', 'EPI_template']
        # this is a fork point
        # NOTE:
        #   this will determine which registration transform to use to warp the functional
        #   outputs and derivatives to template space
        using: [T1_template]

        T1_template:

          # Standard Skull Stripped Template. Used as a reference image for functional registration.
          # This can be different than the template used as the reference/fixed for T1-to-template registration.
          T1w_brain_template_funcreg: /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz

          # Standard Anatomical Brain Image with Skull.
          # This can be different than the template used as the reference/fixed for T1-to-template registration.
          T1w_template_funcreg: /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}.nii.gz

          # Template to be used during registration.
          # It is not necessary to change this path unless you intend to use a non-standard template.
          T1w_brain_template_mask_funcreg: /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}_brain_mask.nii.gz

          # a standard template for resampling if using float resolution
          T1w_template_for_resample: /usr/share/fsl/5.0/data/standard/MNI152_T1_${func_resolution}_brain.nii.gz

        EPI_template:

          # EPI template for direct functional-to-template registration
          # (bypassing coregistration and the anatomical-to-template transforms)
          EPI_template_funcreg:

          # EPI template mask.
          EPI_template_mask_funcreg:

          # a standard template for resampling if using float resolution
          EPI_template_for_resample:

      ANTs_pipelines:

        # Interpolation method for writing out transformed functional images.
        # Possible values: Linear, BSpline, LanczosWindowedSinc
        interpolation: LanczosWindowedSinc

      FNIRT_pipelines:

        # Interpolation method for writing out transformed functional images.
        # Possible values: trilinear, sinc, spline
        interpolation: sinc

        # Identity matrix used during FSL-based resampling of functional-space data throughout the pipeline.
        # It is not necessary to change this path unless you intend to use a different template.
        identity_matrix: /usr/share/fsl/5.0/etc/flirtsch/ident.mat

      apply_transform:

        # options: 'default', 'abcd', 'single_step_resampling_from_stc', 'dcan_nhp'
        # 'default': apply func-to-anat and anat-to-template transforms on motion corrected functional image.
        # 'abcd': apply motion correction, func-to-anat and anat-to-template transforms on each of raw functional volume using FSL applywarp based on ABCD-HCP pipeline.
        # 'single_step_resampling_from_stc': apply motion correction, func-to-anat and anat-to-template transforms on each of slice-time-corrected functional volume using ANTs antsApplyTransform based on fMRIPrep pipeline.
        #   - if 'single_step_resampling_from_stc', 'template' is the only valid option for ``nuisance_corrections: 2-nuisance_regression: space``
        using: single_step_resampling_from_stc


Backend: Add client id to resource models

The client id is an opaque identifier. It is used to filter resources when sending a list request.
Example: Coil is the client but the web app and the phone app have different keys. When Coil wants to read all payments it has created, it will call the list endpoint with a token that corresponds to a grant which includes the list action. The introspection response from the AS must include the client_id that can then be used to filter the payments before returning.

clientId can be added as a column in the grants table.
Add grantId as a foreign key in both the incomingPayments and outgoingPayments tables.
In the auth middleware, add the valid grant (with its clientId) to the grants table (via a new GrantService?). (This means createOutgoingPayment should stop storing grants, and grant locking can be relocated #485)
The grant is already being added to the ctx, so clientId will be accessible to subsequent middlewares via ctx.grant.clientId

This is also needed for quotes to enforce a limit on clients with a grant to read (not read-all) quotes.


enhancement: Unbound Expression equals() use ==

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 45.33 seconds
stream load tsv:          540 seconds loaded<PHONE_NUMBER>9 Bytes, about 132 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size:<PHONE_NUMBER>3 Bytes


The plugin does not work with variable products

The plugin does not work with variable products. Information about the ordered variation is not added to the order.

There is no functionality for variable products in the plugin, this functionality has not been developed. Look in the search for an add-on that expands the functionality for variable products.


[RFC] ProxyRule

Could be nice to brainstorm about how the resource ProxyRule should be defined.
We should be able to :

enable a hostname to target a cargo at specific location using http and https
redirect domain to another eg: www.domain.com -> domain.com
enable tcp / udp to target a specific cargo at specific port
pass extra http header ?

What else should we have ?
This is for now how look look like http and https to target a cargo at specific location:

HttpResources:
  - Name: resource-example
  Kind: ProxyRule
  Config:
    ## Cargo to watch for change
    Watch:
      - Name: getting-started
    Http:
      Domain: get-started.com
      # Type can be public or private
      # If type is private and the cargo is replicated we use a private self signed https
      Type: Public
      # Path location
      Locations:
        - Path: /
          Target:
            Name: getting-started
            Port: 9000

This is the first update that came in my mind
Resources:
  - Name: resource-example
    Kind: ProxyRule
    Config:
      Watch:
        - get-started@global
      Http:
        Domain: get-started.com
        Network: Public
        SslCertificate: /random/path/to/cert
        SslCertificateKey: /random/path/to/key
        SslDhParam: /random/path/to/param
        Locations:
          - Path: /
            Target:
              Key: get-started@global
              Port: 9000


Custom border radius for buttons

The change in blue was merged earlier in another PR. This change completes the button customisation.
Closes #41

@matthewsmawfield This is ready for review/merge. Thanks!


Minor clean-up

I had a look at tests coverage and saw 1-2 things that could be quickly tackled.

Codecov Report

Merging #1112 (335d806) into master (aee77a2) will increase coverage by 0.15%.
The diff coverage is 95.00%.

@@            Coverage Diff             @@
##           master    #1112      +/-   ##
==========================================
+ Coverage   95.98%   96.13%   +0.15%
==========================================
  Files          64       64
  Lines        4107     4092      -15
==========================================
- Hits         3942     3934       -8
+ Misses        165      158       -7

Impacted Files
Coverage Δ

httpie/context.py
87.01% <ø> (+0.76%)
:arrow_up:

tests/test_sessions.py
100.00% <ø> (ø)

httpie/output/writer.py
75.43% <50.00%> (ø)

httpie/core.py
96.12% <100.00%> (+3.92%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update aee77a2...335d806. Read the comment docs.


.user-select In the case of DOM attribute inheritance

DEMO
The above is the effect of the current CSS sequence.
In a multi-level DOM, such as the case where the entire document is prohibited from selecting, all and auto can no longer be combined. Inline styles should work. Should classes without "! Important" be added?

Nice to see 4.5.0

I don't think there's anything we'd change here—nearly all our utilities use this approach and could have similar issues of one class coming before the other. I also think this issue would exist outside our utilities if you used the property-values in custom CSS yourself.
Feel free to re-open and gut check me on that though, @twbs/css-review.


Suggest creating shallow clone to users

Suggest creating a shallow clone with the history truncated

thanks for the PR! without the depth it needs to download 1MB and with the flag around 240 kb. I think it's worth getting the whole history when it's so small. Why would you go for depth=1? Can you tell me any other projects that recommend this?

The rational behind it is not to save on the downloaded bytes. The user is probably not interested in the history of the starter repo and wants to have it as the starting point of his own app. The saved bytes is just a nice to have side effect.
For example TypeScript-Node-Starter does it too.

Sounds good. I can't merge it though because "Some checks haven’t completed yet" which is the continuous-integration/travis-ci. I think because travis only runs on branches that are from my repo, and a merge to master is only allowed if travis succeeds... I'll have to check what I can do about this, if you happen to know, just let me know


[FEEDBACK]:  Navbar Icons Allignment in Mobile View

Type
Site Bug Report
Feedback
The Navbar has a Responsive issue where the search icon is not properly aligned with the options icon.
Screenshots

developer.tbd.website URL
No response
Browser
Chrome
Relevant log output
No response

It is a copy of #901 so close this one

thanks so much. it'll be fixed in #901


Interface for registering Python to Numba type mappings

As part of #665, I'm trying to simplify and document the ways we extend Numba to support new types.
In the case of Python to Numba type mappings, we have two places where this is currently done:

In the function dispatcher (_OverloadedBase.typeof_pyval()): Used to map Python function arguments to Numba types to decide which compiled function to use, or to trigger a new compilation if none can be found.
In the typing context (BaseContext.resolve_value_type()): Used to find the Numba type of globals values accessed from the function being compiled.

Neither of these functions is particularly extensible to new Numba types.  typeof_pval() has a hardcoded mapping in the form of if/else chain, falling back to checking if the Python object has a _numba_type_ attribute.
The resolve_value_type() method has a different hardcoded list of types (longer than typeof_pyval()), and falls back to a dictionary mapping global values to Numba types.

The use case I want to support is a library author wanting a non-standard Python type to map to a particular Numba type during autojit, as well as if they have any global constants of a non-standard Python type.
(I've had some discussion with @sklam about what this API should look like, so I'll follow up with that.)

The @typeof_impl generic function now allows to do this easily :-)


Remove unused ansible directory

I don't think we use this directory anymore, and it seems like it's not worth us maintaining these files when there are changes to the setup.  I would be curious if anyone is using this, but otherwise I think we should remove.
A note for after this is (potentially) merged: When removing directories, sometimes you still end up with an empty directory locally.  To resolve that, you can run git clean -fd which will remove those for you. Alternatively, you can run git clean -fdn for a dry-run to see what would be removed.

There are some additional files related to this directory, which is part of the Vagrant implementation for this repository:

The repo Vagrantfile makes use of the Ansible playbook.
docs/index.md and docs/usage.md both mention the Vagrant setup. These references should be removed if we're deprecating Vagrant entirely.
The bin/cfgov and bin/cfgovd scripts are included as part of the Vagrant setup described in the documentation.

If we're going to completely deprecate Vagrant/Ansible for this repository, then we should clean it all up at once.
I'd be okay with removing all of that from this repo as our internal Ansible repository lets you run this codebase using a Vagrant box, and the current code is likely out of date and won't be updated to include future improvements (like Postgres). It'd be nice if in future we could re-open source as much of the internal Ansible/Vagrant process as possible.

Covered in https://github.com/cfpb/cfgov-refresh/pull/4107


[Question] ColumnCount

Hi, Why ColumnCount doesn't have a BindableProperty?
or did you forgot to implement it? because I need to bind a property from ViewModel for dynamic change(Like Pinching to increase and decrease) to it.

see #53


Tasks failed on Apache Druid

Hello, I'm using helm chart incubator/druid and when I submit any ingestion he fail the task, even when I try to use sample data wikipedia gets the same error:
K8s: minikube
Erro message in task log tab:
Request failed with status code 404
Helm install:
helm install druid -f /home/mateus_oliveira/druid-values.yaml incubator/druid --version 0.2.8 --set overlord.enabled=true
Druid-value.yaml (just enable kafka ingestion)
druid_extensions_loadList: '["druid-histogram", "druid-datasketches", "druid-avro-extensions","druid-kafka-indexing-service","druid-lookups-cached-global", "postgresql-metadata-storage"]'

#22911
The default druid_storage_type value is local. you have to set the your deep storage for the Apache Druid cluster


Unable to create AWS-Transfer (SFTP) server via crossplane

Hi, I'm trying to configure AWS Transfer server using Crossplane (provider-aws version-0.23.0). Please find the all details below

Server config

apiVersion: transfer.aws.crossplane.io/v1alpha1
kind: Server
metadata:
  name: dev-sftp-server-1
  annotations:
    crossplane.io/external-name: dev-sftp-server-1
spec:
  deletionPolicy: Orphan
  forProvider:
    region: ap-southeast-1
    endpointType: public
    loggingRoleRef:
      name: dev-sftp-logging-access-role
    securityPolicyName: TransferSecurityPolicy-2020-06
    tags:
      - key: Name
        value: dev-sftp-server-1
  providerConfigRef:
    name: aws-dev

Results on creating a server using

$ kubectl apply -f sftp-server-test.yaml
server.transfer.aws.crossplane.io/dev-sftp-server-1 created

As per log the server is created, but it's not actually created in the AWS.

Logs from aws-provider-pod

2022-10-18T07:35:14.148Z	DEBUG	provider-aws	Cannot observe external resource	 {
   "controller":"managed/server.transfer.aws.crossplane.io",
   "request":"/dev-sftp-server-1",
   "uid":"149df12d-926e-49c7-a80e-ff593eb468da",
   "version":"219240674",
   "external-name":"dev-sftp-server-1",
   "error":"failed to describe Server: InvalidParameter: 1 validation error(s) found.\n- minimum field size of 19, DescribeServerInput.ServerId.\n",
   "errorVerbose":"InvalidParameter: 1 validation error(s) found.\n- minimum field size of 19, DescribeServerInput.ServerId.\n\nfailed to describe Server\ngithub.com/crossplane/provider-aws/pkg/clients.Wrap\n\t/home/runner/work/provider-aws/provider-aws/pkg/clients/aws.go:978\ngithub.com/crossplane/provider-aws/pkg/controller/transfer/server.(*external).Observe\n\t/home/runner/work/provider-aws/provider-aws/pkg/controller/transfer/server/zz_controller.go:83\ngithub.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile\n\t/home/runner/work/provider-aws/provider-aws/vendor/github.com/crossplane/crossplane-runtime/pkg/reconciler/managed/reconciler.go:681\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler\n\t/home/runner/work/provider-aws/provider-aws/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:298\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/home/runner/work/provider-aws/provider-aws/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:253\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/home/runner/work/provider-aws/provider-aws/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:214\nruntime.goexit\n\t/opt/hostedtoolcache/go/1.16.13/x64/src/runtime/asm_amd64.s:1371"
}
2022-10-18T07:35:14.149Z	DEBUG	controller-runtime.manager.events	Warning	{
   "object":{
      "kind":"Server",
      "name":"dev-sftp-server-1",
      "uid":"149df12d-926e-49c7-a80e-ff593eb468da",
      "apiVersion":"transfer.aws.crossplane.io/v1alpha1",
      "resourceVersion":"219240674"
   },
   "reason":"CannotObserveExternalResource",
   "message":"failed to describe Server: InvalidParameter: 1 validation error(s) found.\n- minimum field size of 19, DescribeServerInput.ServerId.\n"
}

I've followed all closed issues in this repository around the Transfer server setup, this seems to be the new issue. Ple let me know if you need any additional details.

can you upgrade von v0.23 to a later provider ? as minimum to v0.24.1

@haarchri ... I've updated the version to v0.24.1
Still seeing the following error:
Events:
  Type     Reason                         Age               From                                       Message
  ----     ------                         ----              ----                                       -------
  Warning  CannotObserveExternalResource  7s (x6 over 36s)  managed/server.transfer.aws.crossplane.io  failed to describe Server: InvalidParameter: 1 validation error(s) found.
- minimum field size of 19, DescribeServerInput.ServerId.

Any suggestions how to fix this?

@haarchri , here's the spec I'm using to create the server
apiVersion: transfer.aws.crossplane.io/v1alpha1
kind: Server
metadata:
  name: dev-sftp-server-1
  annotations:
    # This will be the actual server name
    crossplane.io/external-name: dev-sftp-server-1
spec:
  deletionPolicy: Orphan
  forProvider:
    region: ap-southeast-1
    domain: S3
    endpointDetails:
      securityGroupIDRefs:
        - name: dev-sftp-security-group
      subnetIDRefs:
        - name: dev-eks-public-subnet1
      vpcIDRef:
        name: dev-vpc
    endpointType: PUBLIC
    identityProviderType: SERVICE_MANAGED
    securityPolicyName: TransferSecurityPolicy-2020-06
    protocols:
      - SFTP
    tags:
      - key: Name
        value: dev-sftp-server-1
  writeConnectionSecretToRef:
    name: dev-sftp-secret
    namespace: default
  providerConfigRef:
    name: aws-dev

All other AWS resources linked here are already created and present, I'm still seeing the DescribeServerInput.ServerId related error, Anything missing in this spec?

Can you remove the external-name ?

Thanks a lot @haarchri , removing the external name did help. but we are seeing this unexpected error
apiVersion: transfer.aws.crossplane.io/v1alpha1
kind: Server
metadata:
  name: dev-sftp-server-1
spec:
  deletionPolicy: Orphan
  forProvider:
    region: ap-southeast-1
    domain: S3
    endpointDetails:
      securityGroupIDRefs:
        - name: dev-sftp-security-group
      subnetIDRefs:
        - name: dev-eks-public-subnet1
      vpcIDRef:
        name: dev-vpc
      addressAllocationIDRefs:
        - name: dev-sftp-eip
    endpointType: VPC
    identityProviderType: SERVICE_MANAGED
    securityPolicyName: TransferSecurityPolicy-2020-06
    protocols:
      - SFTP
    tags:
      - key: Name
        value: dev-sftp-server-1
  writeConnectionSecretToRef:
    name: dev-sftp-secret
    namespace: default
  providerConfigRef:
    name: aws-dev

Seeing error while setting AddressAllocationRefIDs
Status:
  At Provider:
  Conditions:
    Last Transition Time:  2022-10-18T16:04:29Z
    Reason:                Creating
    Status:                False
    Type:                  Ready
    Last Transition Time:  2022-10-18T16:04:29Z
    Message:               create failed: cannot create Server in AWS: InvalidRequestException: AddressAllocationIds cannot be set in CreateServer
    Reason:                ReconcileError
    Status:                False
    Type:                  Synced
Events:
  Type     Reason                        Age                    From                                       Message
  ----     ------                        ----                   ----                                       -------
  Warning  CannotCreateExternalResource  4m33s (x133 over 50m)  managed/server.transfer.aws.crossplane.io  cannot create Server in AWS: InvalidRequestException: AddressAllocationIds cannot be set in CreateServer

Seeing the same error incase I don't set the AddressAllocationIds | AddressAllocationRefIDs.
How should we configure this?

okay looks like we need to add a fix for the provider - aws docs says:
"This property can only be set when EndpointType is set to VPC and it is only valid in the UpdateServer API."

we need to remove this code part: https://github.com/crossplane-contrib/provider-aws/blob/master/pkg/controller/transfer/server/setup.go#L131-L134 implemente isUp2Date check and inject AddressAllocationIds in preUpdate


ISSUE? ica.plot_components() requires ch_type when picks is not None

With my Brainvision data, these work:
ica.plot_components()

ica.plot_components(picks=[1,2], ch_type="eeg")

but this doesn't:
ica.plot_components(picks=[1,2])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-16-64eeca493160> in <module>()
----> 1 ica.plot_components(picks=[1,2])

/home/FB/fb05/sassenha/tools/mne-working/mne-python/mne/preprocessing/ica.py in plot_components(self, picks, ch_type, res, layout, vmin, vmax, cmap, sensors, colorbar, title, show, outlines, contours, image_interp, head_pos)
   1352                                    outlines=outlines, contours=contours,
   1353                                    image_interp=image_interp,
-> 1354                                    head_pos=head_pos)
   1355
   1356     def plot_sources(self, inst, picks=None, exclude=None, start=None,

/home/FB/fb05/sassenha/tools/mne-working/mne-python/mne/viz/topomap.py in plot_ica_components(ica, picks, ch_type, res, layout, vmin, vmax, cmap, sensors, colorbar, title, show, outlines, contours, image_interp, head_pos)
    690
    691     data_picks, pos, merge_grads, names, _ = _prepare_topo_plot(ica, ch_type,
--> 692                                                                 layout)
    693     pos, outlines = _check_outlines(pos, outlines, head_pos)
    694     if outlines not in (None, 'head'):

/home/FB/fb05/sassenha/tools/mne-working/mne-python/mne/viz/topomap.py in _prepare_topo_plot(inst, ch_type, layout)
     56
     57         if len(picks) == 0:
---> 58             raise ValueError("No channels of type %r" % ch_type)
     59
     60         if layout is None:

ValueError: No channels of type 'mag'

Am I missing something, or is this a bug?

This looks like a bug. Btw. didn't we ant to guess the channel type at some point? Why do we still need to pass the channel type? @wmvanvliet @mainakjas @agramfort

Closed for my bad PR at #2336 I guess.


Added mesos-resource-provider-sdk in client libraries.

I wrote a resource provider sdk in Go for mesos, which worked fine on my ARM chips.
Also, since the mesos-framework-sdk is no longer maintained, I forked it and added some features.
I'm not sure if it's OK to replace the link in the doc.

@carlonelong Thanks for your contribution! This PR has been merged through https://github.com/apache/mesos/commit/fb1695f93acad12f07634423e205fad1384dde1e


map editor

Hi
when I do SLAM with mapping.launch and rviz looks pretty well so
In the next window that pops up after pressing y, the inverted screen for the center line was not created properly, so I thought the border of the map was not clear, so I tried to work with png directly through the map editor, However, the appearance of the saved png file was also saved as an inverted map, different from the appearance of the map in rviz. It looks like GIMP work will be difficult with this file. Is the result correct?

Hi, can you show the map image?

This is what the png file looks like when saving the map using
roslaunch stack_master mapping.launch map_name:= racecar_version:=
after the actual track and slam.
and also when I doing
roslaunch map_editor map_editor.launch map_name:=PLACE_DATE_VN map_editor_mapping:=True
This is not saved as a pgm file for direct map editing, but as a png file.

I see. Using black tubes is difficult for the LiDAR as it does not reflect as much (see: https://arxiv.org/pdf/2309.10504).
Hence when mapping you need to drive multiple times to get a good map. Probably the SLAM did not get a nice occupancy grid, that's why the part with the white wall is visible in the png but it has problems with the black only sections.
Different things you can do:

Adapt the SLAM config here for bad reflectivity based on the docs and redo the mapping
Redo the mapping and drive slowly and multiple times
You don't need tubes, you can draw an arbitrary map in GIMP. I.e. you could just "finish" the current png. This will not represent the tubes accurately, but then again, technically you don't need them, as localisation is performed with the .pbstream and the track is only needed to compute the racing line

Should all be saved as png. Check out some additional info in the readme: https://github.com/ForzaETH/race_stack/tree/main/f110_utils/nodes/map_editor


Always return results in Mb/s

This PR aims to fix issue #6 in which speed result units may be incorrectly reported.

Please also delete the functions getJustifiedSpeed and getSpeedUnit. They are no longer used, so we should not leave them lying around.
LGTM


Unable to import appliance!

Im new to pyvbox
I tried to import an ovf file but get this error
virtualbox.library.OleErrorFail: 0x80004005 (Cannot interpret appliance without reading it first (call read() before interpret()))
this is the code

import virtualbox
vbox = virtualbox.VirtualBox()
apl = vbox.create_appliance()
apl.read('~/Documents/ubuntu.ova')

why before reading completion , interpret is called!

I've found that the underlying virtualbox api that pyvbox relies on is extremely temperamental about the path to the file. I suggest running the path through os.path.absdir() and strip() . Even stray characters after os.path.absdir will cause this error.

@markhuber I agree.  The tilde may be the cause of the issue in the posted example. Cheers

This is reaaally old, but just to clarify in case anyone comes to this bugreport:
.read('~/Documents/ubuntu.ova') will not find the file (as it really does not exist), therefore the error.
What you really whated to do is api.read(os.path.expanduser('~/Documents/ubuntu.ova').


Incorrect query for spending_by_practice endpoint

Incorrect SQL is generated for spending_by_practice when the query...

is for a product or presentation
covers multiple organisations
includes a date

There is a test for this in f81d688d, and a fix in 98920090.
I'm fairly sure that our JS code never calls this endpoint with a date.  (See chart_utils.js.)
But somebody has been, and moreover has been for multiple organisations:
$ grep spending_by_practice nginx-access.log | grep date | grep -E 'org=\w+,' | wc -l
2402

(nginx-access.log goes back to 2018-05-13.)
But nobody has (recently) made any API calls that satisfy all three bullet point conditions:
$ grep spending_by_practice nginx-access.log | grep date | grep -E 'org=\w+,' | grep -E 'code=\w{10,}' | wc -l
0

We should review and deploy the fix ASAP.  And we should think about how to communicate this to users of the API.

Looking at the recent logs I get the feeling that somebody has been trying to reverse engineer the API -- there are lots of queries for things that the API will ignore, eg date=2017-12-01&date=2017-01-02.
And the requests initially come from the browser, while later they come from localhost, and then they come from http://healthcaredataviz.azurewebsites.net (eg this page).
Would be worth getting in touch with whoever runs that site (probably related to www.healthcaredataviz.com) to find out about their API usage.

This was fixed in #851.


Block request attempts

This patch prevents connection attempts that aren't addressed by their own patches.
Fixes #1008, #995, and #1049

Sorry for all the PRs right before a major version update!
Let me know if it makes more sense to relocate the patch, this is just what makes the most sense to me.
Extra Info:  I removed the d3 script from the chrome://discards/ page since it doesn't change the current behavior (the graph page is blank) and stops the browser from making unneeded requests.  If there is ever reason to need that functionality restored, the URL could be replaced with another from a non-Google domain like d3's hosted version or CloudFlare's.
Google has more info about the graph view and their reasoning for not bundling the script here:  https://chromium.googlesource.com/chromium/src.git/+/master/chrome/browser/resources/discards/graph_view.md

its not that it is wrong, its just that it just gives you more work to update for the new release.

This will need the first line offset changed from 60 to 78 for #1159.  I'll update the commit in a bit.


"unsupported operand type(s) for -: 'NoneType' and 'NoneType'"

I am not able to install the Extension on SLES 15. Is this a known issue?
Set-AzVMAEMExtension : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure
when processing extension 'AzureEnhancedMonitorForLinux'. Error message: "unsupported operand type(s) for -:
'NoneType' and 'NoneType'"
More information on troubleshooting is available at https://aka.ms/vmextensionlinuxtroubleshoot '
ErrorCode: VMExtensionProvisioningError
ErrorMessage: VM has reported a failure when processing extension 'AzureEnhancedMonitorForLinux'. Error message:
"unsupported operand type(s) for -: 'NoneType' and 'NoneType'"
More information on troubleshooting is available at https://aka.ms/vmextensionlinuxtroubleshoot
ErrorTarget:
StartTime: 13.12.2019 20:51:26
EndTime: 13.12.2019 20:52:09
OperationID: 89d2cc20-e0dd-481a-9ce5-03893724a3af
Status: Failed
At line:1 char:1

Set-AzVMAEMExtension -ResourceGroupName xxxxxx -VMName xxxxx ...

  + CategoryInfo          : CloseError: (:) [Set-AzVMAEMExtension], ComputeCloudException
  + FullyQualifiedErrorId : Microsoft.Azure.Commands.Compute.SetAzureRmVMAEMExtension

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3ea8dbf7-4305-1342-c073-8c7b20564fd3
Version Independent ID: a3692496-1cf4-854c-94da-419058bd3e60
Content: Azure Virtual Machines deployment for SAP NetWeaver
Content Source: articles/virtual-machines/workloads/sap/deployment-guide.md
Service: virtual-machines-linux
GitHub Login: @MSSedusch
Microsoft Alias: sedusch

@pbaumgartner please provide a link to the document that you are following so that we can better assist.

@pbaumgartner please provide a link to the document that you are following so that we can better assist.

I execute Set-AzVMAEMExtension -ResourceGroup xxxx -VMname xxxx -OSType Linux as per https://docs.microsoft.com/en-us/powershell/module/az.compute/set-azvmaemextension?view=azps-3.1.0 and https://launchpad.support.sap.com/#/notes/2015553 and https://docs.microsoft.com/en-us/azure/virtual-machines/workloads/sap/deployment-guide?toc=%2Fazure%2Fvirtual-machines%2Flinux%2Ftoc.json#update-the-configuration-of-azure-extension-for-sap and https://docs.microsoft.com/en-us/azure/virtual-machines/workloads/sap/deployment-guide#d98edcd3-f2a1-49f7-b26a-07448ceb60ca

@MSSedusch please add your comments

@pbaumgartner the extension is currently not supported for SLES 15. We are working on a new version. We will update the guide once it is available.

Very bad, as this is a prerequisite for having a supported SAP System in Azure, where SLES15 is already certified. Waiting for your news, is there a time expected?

we should be able to release it in Q1/CY20. Sorry for the inconvenience.

@MSSedusch   Assigned this issue to you.
Please close the issue once the new extension is released.

We are closing this but once it is supported the doc will be updated. See the above comments for our ETA
#please-close

@mimckitt - would it be possible to keep this issue open until it's actually fixed as per the comment form @jakaruna-MSFT?  Closing this now and updating documentation (in an unknown and un-referenced location) at a future point is harder to track and doesn't give an easy way of people to know when it's been fixed - or even when it's actively being worked upon.

I will update this issue once we extension is released. @mimckitt can you reopen? thanks

Is there an update on this?

not yet. I hope to have an update in a few weeks.

Hi @MSSedusch Any update?

@jahobson75 the new Extension and SAP Host Agent are available. We will publish guidance on how to install and troubleshoot the new extension in the next few days.
PublisherName: Microsoft.AzureCAT.AzureEnhancedMonitoring
Type: MonitorX64Windows

@jahobson75 the new Extension and SAP Host Agent are available. We will publish guidance on how to install and troubleshoot the new extension in the next few days.
PublisherName: Microsoft.AzureCAT.AzureEnhancedMonitoring
Type: MonitorX64Windows

Very good news, will you provide that link here?

@pbaumgartner yes, the guidance will be part of the deployment guide. https://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide
But I will update this issue once the article has been updated.

The updated SAP deployment guide went live yesterday.
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide
The following chapter describe how you can install the new extension using PowerShell or Azure CLI
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide#2ad55a0d-9937-4943-9dd2-69bc2b5d3de0
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide#c8749c24-fada-42ad-b114-f9aae2dc37da

The updated SAP deployment guide went live yesterday.
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide
The following chapter describe how you can install the new extension using PowerShell or Azure CLI
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide#2ad55a0d-9937-4943-9dd2-69bc2b5d3de0
http://docs.microsoft.com/azure/virtual-machines/workloads/sap/deployment-guide#c8749c24-fada-42ad-b114-f9aae2dc37da

@MSSedusch thank you for sharing, i installed the new extension, what was successful, but in SAP ST06 it's still visible as FALSE! and in /var/lib/AzureEnhancedMonitor nothing is visible. no file or so since installation.

Did you also update your SAP Host Agent to PL47? It is not yet included in the documentation. I added this information a couple of hours ago and will be part of the published deployment guide tomorrow.

Did you also update your SAP Host Agent to PL47? It is not yet included in the documentation. I added this information a couple of hours ago and will be part of the published deployment guide tomorrow.

good point, hope it's added soon, it's working with the latest version 47.

We are currently cleaning up old issues and closing out items that are greater than 90 days old. If an issue is still present, please open a new feedback item on the document so we can prioritize correctly.
#please-close


Fix map

Attempt to fix sourcemap ...
please test this 🥴 :D
fix #3 and superseed #40

Thanks! let me try this evening

Works like a charm!

I'm going to publish a pre-release. This shouldn't significantly degrade build performance in production.

Just confirmed no performance degradation on my production builds.
Released as v0.5.0


Installing custom genome failure with gff3

Hello,
I am trying to build a custom genome for XenTro10 and I believe I get pretty far in the process, it ran for a few hours and then failed. I see where it failed (I bolded it) but I have no idea what that means and how to fix it. (I am using the newest version of bcbio)
I found my fasta file and gttf3 files using Xenbase (http://ftp.xenbase.org/pub/Genomics/JGI/Xentr10.0/)
Here is my run script
#!/bin/bash
#SBATCH -p general
#SBATCH --job-name=xtrop10_build
#SBATCH -c 10
#SBATCH -t 24:00:00
#SBATCH --mem=100G
#SBATCH -e xtrop10_build.err
export PATH=/proj/conlonlb/bcbio/anaconda/bin:/proj/conlonlb/bcbio/tools/bin:$PATH
bcbio_setup_genome.py
-f /proj/conlonlb/XENTR_10.0_genome.fasta.gz
-g /proj/conlonlb/XENTR_10.0_GCF.gff3
-i bwa star seq
-n XenTro -b XenBase10_0 --buildversion XenBase10
Here is my .err file
--2020-07-01 11:21:56--  https://github.com/chapmanb/cloudbiolinux/archive/master.tar.gz
Resolving github.com (github.com)... <IP_ADDRESS>
Connecting to github.com (github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master [following]
--2020-07-01 11:21:56--  https://codeload.github.com/chapmanb/cloudbiolinux/tar.gz/master
Resolving codeload.github.com (codeload.github.com)... <IP_ADDRESS>
Connecting to codeload.github.com (codeload.github.com)|<IP_ADDRESS>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [application/x-gzip]
Saving to: ‘STDOUT’
 0K ........ ........ ........ ........ ........ ........ 3.14M

3072K ........ ........ ........ .......                    2.24M=1.8s
2020-07-01 11:21:58 (2.71 MB/s) - written to stdout [5189916]
[bwa_index] Pack FASTA... 9.70 sec
[bwa_index] Construct BWT for the packed sequence...
[BWTIncCreate] textLength=2902567198, availableWord=216235256
[BWTIncConstructFromPacked] 10 iterations done. 99999998 characters processed.
[BWTIncConstructFromPacked] 20 iterations done. 199999998 characters processed.
[BWTIncConstructFromPacked] 30 iterations done. 299999998 characters processed.
[BWTIncConstructFromPacked] 40 iterations done. 399999998 characters processed.
[BWTIncConstructFromPacked] 50 iterations done. 499999998 characters processed.
[BWTIncConstructFromPacked] 60 iterations done. 599999998 characters processed.
[BWTIncConstructFromPacked] 70 iterations done. 699999998 characters processed.
[BWTIncConstructFromPacked] 80 iterations done. 799999998 characters processed.
[BWTIncConstructFromPacked] 90 iterations done. 899999998 characters processed.
[BWTIncConstructFromPacked] 100 iterations done. 999999998 characters processed.
[BWTIncConstructFromPacked] 110 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 120 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 130 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 140 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 150 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 160 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 170 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 180 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 190 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 200 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 210 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 220 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 230 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 240 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 250 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 270 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 280 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 290 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 300 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 310 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 320 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 330 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 340 iterations done.<PHONE_NUMBER> characters processed.
[BWTIncConstructFromPacked] 350 iterations done.<PHONE_NUMBER> characters processed.
[bwt_gen] Finished constructing BWT in 359 iterations.
[bwa_index] 985.42 seconds elapse.
[bwa_index] Update BWT... 8.26 sec
[bwa_index] Pack forward-only FASTA... 5.92 sec
[bwa_index] Construct SA from BWT and Occ... 422.89 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index -a bwtsw XenBase10_0.fa
[main] Real time: 1441.218 sec; CPU: 1432.215 sec
INFO    2020-07-01 12:45:30     CreateSequenceDictionary
********** NOTE: Picard's command line syntax is changing.

********** For more information, please see:
********** https://github.com/broadinstitute/picard/wiki/Command-Line-Syntax-Transition-For-Users-(Pre-Transition)

********** The command line looks like this in the new syntax:

**********    CreateSequenceDictionary -REFERENCE /proj/conlonlb/bcbio/genomes/XenTro/XenBase10_0/seq/XenBase10_0.fa -OUTPUT /proj/conlonlb/bcbio/genomes/XenTro/XenBase10_0/seq/XenBase10_0.dict

12:45:32.050 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/proj/conlonlb/bcbio/anaconda/share/picard-2.23.0-0/picard.jar!/com/intel/gkl/native/libgkl_compression.so
[Wed Jul 01 12:45:32 EDT 2020] CreateSequenceDictionary OUTPUT=/proj/conlonlb/bcbio/genomes/XenTro/XenBase10_0/seq/XenBase10_0.dict REFERENCE=/proj/conlonlb/bcbio/genomes/XenTro/XenBase10_0/seq/XenBase10_0.fa    TRUNCATE_NAMES_AT_WHITESPACE=true NUM_SEQUENCES=2147483647 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false
[Wed Jul 01 12:45:32 EDT 2020] Executing as<EMAIL_ADDRESS>on Linux 3.10.0-1062.el7.x86_64 amd64; OpenJDK 64-Bit Server VM 1.8.0_192-b01; Deflater: Intel; Inflater: Intel; Provider GCS is not available; Picard version: 2.23.0
[Wed Jul 01 12:45:38 EDT 2020] picard.sam.CreateSequenceDictionary done. Elapsed time: 0.11 minutes.
Runtime.totalMemory()=1592262656
Building a SMALL index
Traceback (most recent call last):
File "/proj/conlonlb/cloudbiolinux/utils/prepare_tx_gff.py", line 846, in
main(args.org_build, args.gtf, args.fasta, genome_dir, args.cores, args)
File "/proj/conlonlb/cloudbiolinux/utils/prepare_tx_gff.py", line 296, in main
gtf_file = clean_gtf(gtf_file, genome_fasta)
File "/proj/conlonlb/cloudbiolinux/utils/prepare_tx_gff.py", line 362, in clean_gtf
if line.split()[0].strip() not in fa_names:
IndexError: list index out of range
Traceback (most recent call last):
File "/proj/conlonlb/bcbio/anaconda/bin/bcbio_setup_genome.py", line 313, in
subprocess.check_call(cmd.format(**locals()), shell=True)
File "/proj/conlonlb/bcbio/anaconda/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/proj/conlonlb/bcbio/anaconda/bin/python3.6 /proj/conlonlb/cloudbiolinux/utils/prepare_tx_gff.py --buildversion XenBase10 --cores 1 --genome-dir /proj/conlonlb/bcbio/genomes --gtf /proj/conlonlb/bcbio/genomes/XenTro/XenBase10_0/rnaseq/ref-transcripts.gtf XenTro XenBase10_0' returned non-zero exit status 1.

Hello @drlaurenwasson!
Thanks for the detailed report.
I think it is because the input annotation is gff3 rather than gtf.
For gff3 there is a flag in bcbio_setup_genome.py:
bcbio_setup_genome.py \
-c 15 \
-f XENTR_10.0_genome.fasta.gz \
--gff3 \
-g XENTR_10.0_GCF.gff3 \
-i bwa star seq \
-n XenTro -b XenBase10_0 --buildversion XenBase10

Unfortunately, this fails with a bug.
I managed to convert with gffread XENTR_10.0_GCF.gff3 -T -o XENTR_10.0_GCF.gtf (gffread is installed with bcbio).
Could you try this conversion and then:
bcbio_setup_genome.py \
-c 15 \
-f XENTR_10.0_genome.fasta.gz \
-g XENTR_10.0_GCF.gtf \
-i bwa star seq \
-n XenTro -b XenBase10_0 --buildversion XenBase10

Sergey

Hi Sergey,
This installation also finished successfully for me! Thank you so much


Import error no module named pyproj._datadir in pyproj

Hi,
I'm getting below error while running python .exe file.

its working fine when i run .py file directly in cmd. but failing when i run .exe file.
Python Version: 3.7
pyproj : 2.1.3
geopandas :0.4
Any help on this much appreciated.
Thanks,
Mani

Mind sharing installation method & windows version?

Hi Snowman,
i used pip install pyproj.
Below for installed packages.
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org geopandas
Requirement already satisfied: geopandas in c:\python37\lib\site-packages (0.4.1)
Requirement already satisfied: fiona in c:\python37\lib\site-packages (from geopandas) (1.8.6)
Requirement already satisfied: shapely in c:\python37\lib\site-packages (from geopandas) (1.6.4.post1)
Requirement already satisfied: pandas in c:\python37\lib\site-packages (from geopandas) (0.24.2)
Requirement already satisfied: pyproj in c:\python37\lib\site-packages (from geopandas) (2.1.3)
Requirement already satisfied: six>=1.7 in c:\python37\lib\site-packages (from fiona->geopandas) (1.12.0)
Requirement already satisfied: cligj>=0.5 in c:\python37\lib\site-packages (from fiona->geopandas) (0.5.0)
Requirement already satisfied: attrs>=17 in c:\python37\lib\site-packages (from fiona->geopandas) (19.1.0)
Requirement already satisfied: click>=4.0 in c:\python37\lib\site-packages (from fiona->geopandas) (7.0)
Requirement already satisfied: click-plugins>=1.0 in c:\python37\lib\site-packages (from fiona->geopandas) (1.1.1)
Requirement already satisfied: munch in c:\python37\lib\site-packages (from fiona->geopandas) (2.3.2)
Requirement already satisfied: python-dateutil>=2.5.0 in c:\python37\lib\site-packages (from pandas->geopandas) (2.8.0)
Requirement already satisfied: pytz>=2011k in c:\python37\lib\site-packages (from pandas->geopandas) (2018.9)
Requirement already satisfied: numpy>=1.12.0 in c:\python37\lib\site-packages (from pandas->geopandas) (1.16.2)
Thanks,
Mani

I ran into the same problem after packing my project with pyinstaller. I don't really know how to debug this so any help will be very appreciated :)

I've never used pyinstaller.  Here's what pyinstaller's documentation says about Cython Support.

https://stackoverflow.com/questions/55824830/i-get-error-no-module-named-pyproj-datadir-after-i-made-py-to-exe-with-py
I had the same problem but maybe you can try this..

@Manikandankasi01 did https://github.com/pyproj4/pyproj/issues/273#issuecomment-486249484 or https://github.com/pyproj4/pyproj/issues/273#issuecomment-486517407 resolve your issue?

Same error on a notebook.

Almost the same error on a notebook.

I open an other issue for unix users ?

my path to PROJ.4 wasn't the right one. After setting it to for me /usr/share/proj, it works.

Feel free to reopen or continue the thread if the suggestions provided didn't provide the solution.

Anyone got a solution for this please ?

Anyone got a solution for this please ?

Did you try out either of these solutions: https://github.com/pyproj4/pyproj/issues/273#issuecomment-486249484 or https://github.com/pyproj4/pyproj/issues/273#issuecomment-486517407?

Anyone finding this via a search: pyinstaller v4 has the hooks specifically to address this pyproj issue. All working for me now.

Has something changed in the latest pyproj 3.0.0 that causes trouble again? When runnin pyinstaller I am getting
  File "pyproj\crs\__init__.py", line 7, in <module>
  File "pyproj/_crs.pyx", line 1, in init pyproj._crs
ModuleNotFoundError: No module named 'pyproj._compat'

but I am new to pyinstaller as well, so it might be my fault :-(

I am wondering if pyinstaller needs to add hooks as mentioned here: https://github.com/pyproj4/pyproj/issues/273#issuecomment-717738650?


Install.sh is dowbloading too much

Please update install.sh so that it doesn't download all the pokemon, it's downloading waaay to much and take lot of place.
Thank you

It's in fact not downloading but expanding all the 651gifs to colored ascii art which takes useless amounts of Hard Drive space.
I'll add a full-install option and leave the default one just as a dependencies check.


Reading req.body has huge performance lack

So, I am currently coding with iron (the web framework), since it is web stuff I want to read the users request body. And that's where I ran into damn big issues.
Wether I use io::Read or io::BufReader, wether I use .read(), .read_to_end() or .read_to_string(), my benchmark went from ~290k (without read) down to ~112 requests per second.
Example code:
fn hello(req: &mut Request) -> IronResult<Response> {
    let mut body = String::new();
    let mut bytes = Vec::new();
    let mut lbytes = vec![0;100];

    // NOTICE: I checked, those reads are successful and return the correct value.
    req.body.read_to_end(&mut bytes).unwrap();
    // Or
    req.body.read(&mut lbytes).unwrap();
    // Or
    req.body.read_to_string(&mut body).unwrap();

    Ok(Response::with((status::Ok, "Hello World!")))
}

Does anyone have a workaround for the same issues? Is there maybe a duplicate question I didn't found? Is this issue known?

@plumenator I did a POST request with a normal user login, about 40bytes. I haven't been working in rust for a while, so yeah. For more research I need some time.

@plumenator I tested your code, and it works now. I really have no idea what I did wrong, cause it works with the same code now. Reached ~320k requests per second again. Thanks.

Good to know!


BedrockEmbeddings BUG: Default values for dim and norm in embed_documents() overwrite model_kwargs for Titan Embedding v2 model

The changes introduced in PR #39 make impossible for one to set dimensions or normalize parameters in the model_kwargs of BedrockEmbeddings class.
Example below:
from langchain_aws import BedrockEmbeddings

# The options set here are overwritten
model_kwargs = {
    "dimensions": 256,
    "normalize": False
}

embeddings = BedrockEmbeddings(model_id="amazon.titan-embed-text-v2:0",
                               region_name="eu-central-1", model_kwargs=model_kwargs)

text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."
embedded = embeddings.embed_documents(texts=[text])

print(embedded[0])         # Vector is normalized because of the norm = True default in embed_documents()
print(len(embedded[0]))    # Prints a length of 1024 because of the dim = 1024 default in embed_documents()

This is problematic when using the BedrockEmbeddings with a vector store:
from langchain_aws import BedrockEmbeddings
from langchain_postgres.vectorstores import PGVector
import os

DB_USER = os.getenv('PGUSER')
DB_PASSWORD = os.getenv('PGPASSWORD')
DB_HOST = os.getenv('PGHOST')
DB_PORT = os.getenv('PGPORT')
DB_NAME = os.getenv('PGDATABASE')

model_kwargs = {
    "dimensions": 256,
    "normalize": False
}

connection = f"postgresql+psycopg://{DB_USER}:{DB_PASSWORD}@{DB_HOST}:{DB_PORT}/{DB_NAME}"

text = "Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua."

embeddings = BedrockEmbeddings(model_id="amazon.titan-embed-text-v2:0",
                               region_name="eu-central-1", model_kwargs=model_kwargs)

pgvector_store = PGVector(
    connection=connection, embeddings=embeddings)

pgvector_store.add_texts(texts=[text])   # add_texts() calls embed_documents() and model_kwargs are overwritten again

Expected behavior is model_kwargs not to be overwritten by any method unless explicit values are provided.
EDIT: Even if the embeddings are created separately from the vector store e.g:
embedded_texts = embeddings.embed_documents(
        texts=texts, dim=512)

vector_store.add_embeddings(
        texts=texts,
        embeddings=embedded_texts
)

When the chain is invoked with the vector store retriever it throws an error for different dimension vectors since it creates a query vector with the default 1024 dimensions:
retriever = vector_store.as_retriever()
    chain = (
        {
            "context": RunnableLambda(get_retriever_query_dict) | retriever,
            "question": itemgetter("question")
        }
        | prompt
        | model
        | StrOutputParser()
    )
chain.invoke({"question": question})

[SQL: SELECT langchain_pg_embedding.id AS langchain_pg_embedding_id, langchain_pg_embedding.collection_id AS langchain_pg_embedding_collection_id, langchain_pg_embedding.embedding AS langchain_pg_embedding_embedding, langchain_pg_embedding.document AS langchain_pg_embedding_document, langchain_pg_embedding.cmetadata AS langchain_pg_embedding_cmetadata, langchain_pg_embedding.embedding <=> %(embedding_1)s AS distance

sqlalchemy.exc.DataError: (psycopg.errors.DataException) different vector dimensions 512 and 1024

@trendafil-gechev
Thanks for opening this issue. The earlier PR seems to have deviated from the the base Embeddings interface. You are right, we should not have allowed adding these options directly in the embed methods, rather applied them through the model_kwargs or directly via the class.
@bigbernnn
Is there ever need for different embedding dimensions from within a single embeddings class?


Add acceptance test for doi manuscript pdf service

What?
The pass-ui interaction usage of the DOI service which returns available pdfs should have an acceptance test.
Why?
We missed updating the calls to the doi service for CSRF protection by the ui because of a lack of testing.
How?
Acceptance Criteria
A dedicated acceptance test.
Related Issues

@markpatton Does this test cover the case? https://github.com/eclipse-pass/pass-acceptance-testing/blob/776d26909ea1be4366b342214985ce5a750936b9/tests/dashboardTests.js#L26

@markpatton or maybe you are referring to the Files step where the calls to the DOI service are made for manuscript?

I also wanted to confirm a doubt I have.  I ran a test locally using pass-docker.  I entered DOI 10.1039/c7an01256j.  I got to the Files step, and I see this request being made http://localhost:8080/doi/manuscript?doi=10.1039/c7an01256j, and it returns:

    {
        "url": "https://europepmc.org/articles/pmc6759371?pdf=render",
        "repositoryLabel": "PubMed Central - Europe PMC",
        "type": "application/pdf",
        "source": "Unpaywall",
        "name": "pmc6759371?pdf=render",
        "isBest": true
    },
    {
        "url": null,
        "repositoryLabel": null,
        "type": "application/pdf",
        "source": "Unpaywall",
        "name": null,
        "isBest": false
    }
]

Based on the code I see in the UI, I think the UI should show the pmc6759371?pdf=render manuscript PDF, but it displays this message instead: We could not find any existing open access copies for your manuscript/article. Please upload your own copy.

@rpoet-jh Right, I was referring the Files step in particular. I also have observed the same behavior of the service not seeming to work. I had meant to make a separate ticket.


may be bug options->FIELD use double\single quotes

Dear Dev
I use according to the documentation single quotes
$options->FIELD_ENCLOSURE='"' and an error appears,

ValueError: fgetcsv(): Argument #3 ($separator) must be a single character

but if I use double quotes
$options->FIELD_ENCLOSURE="\""; - there is no error

Can you propose a PR with a failing test for this please?


Issue when pressing escape key

Issues when we go from fullscreen to windowed mode by pressing the escape key on the keyboard, rather than fullscreen button present on the image-viewer.
Because in your toggleFullscreen() method it just reverses the value of 'fullscreen' value. So when we press escape key in fullscreen mode, it takes us to windowed mode but the value of fullscreen value stays the same(since toggleFullscreen() wasn't invoked), and hence it takes 2 clicks on the fullscreen button to take to fullscreen again.

Escape key event is not fired in fullscreen mode. So gotta find some other way around.


Lift _count outside of the select block

Overview
For cases where I am interested in _count as an extra additional column, the select API placement makes it less ergonomic as I need to manually specify all of the remaining columns interesting for me from the model (and nested models). This is probably more related to the select semantics but count here is in the scope of the findMany operation, so opting out from all columns when specifying an "extra" _count creates a struggle (think about a table with 20 cols which you need to retype in select block).
Originally posted by @ssukienn in https://github.com/prisma/prisma/issues/15069#issuecomment-1539896123

(Closed because _count is already available in include, which solves the problem)


Bug: unable to add edges while using Kuzu's LlamaIndex PropertyGraphIndex integration

Kùzu version
v0.6.0
What operating system are you using?
MacOS 14.4.1 (23E224)
What happened?
I was creating a graph using LlamaIndex's PropertyGraphIndex integration with Kuzu. I was trying to add nodes and edges to the graph (not using PropertyGraphIndex.from_documents, but by manually creating nodes/relations and upserting them to the graph). I was able to add nodes to the graph, but none of the edges were showing up in the graph. Upon inspecting the code inside KuzuPropertyGraphStore.upsert_relations:
def upsert_relations(self, relations: List[Relation]) -> None:
        for rel in relations:
            if self.has_structured_schema:
                src, _, dst = utils.lookup_relation(rel.label, self.relationship_schema)
            else:
                src, dst = "Entity", "Entity"

            rel_tbl_name = f"LINKS_{src}_{dst}"
            self.connection.execute(
                f"""
                MATCH (a:{src} {{id: $source_id}}),
                        (b:{dst} {{id: $target_id}}),
                        (c:Chunk {{id: $triplet_source_id}})
                MERGE (a)-[r:{rel_tbl_name} {{label: $label}}]->(b)
                    SET r.triplet_source_id = $triplet_source_id
                MERGE (c)-[:LINKS_Chunk_{src} {{label: "MENTIONS"}}]->(a)
                MERGE (c)-[:LINKS_Chunk_{dst} {{label: "MENTIONS"}}]->(b)
                """,
                parameters={
                    "source_id": rel.source_id,
                    "target_id": rel.target_id,
                    "triplet_source_id": rel.properties.get("triplet_source_id"),
                    "label": rel.label,
                },
            )

I found that the code to add edges relies on the fact that there should be Chunk nodes inserted into the graph, and if they do not exist (which is the situation for me) then the edge does not get added.
Ideally, in case Chunk nodes are not found but we do supply nodes between which we would like an edge to be created, it should not be conditional on a Chunk node also being present.
Are there known steps to reproduce?
A minimal example which should be enough to reproduce the issue:
import kuzu
from llama_index.llms.openai import OpenAI
from llama_index.core.graph_stores import EntityNode, Relation

db = kuzu.Database("/tmp/demo_db")
graph_store = KuzuPropertyGraphStore(db=db)

index = PropertyGraphIndex.from_existing(property_graph_store=kuzu_property_graph_store,llm=OpenAI(....<params>....))
graph_store.upsert_nodes([EntityNode(name="node1", label="Entity", properties={})])
graph_store.upsert_nodes([EntityNode(name="node2", label="Entity", properties={})])
graph_store.upsert_relations([Relation(source_id="node1", target_id="node2", label="Relation", properties={})])

After running the the code, use the Kuzu Explorer and point it to tmp/demo_db to see the graph. the query MATCH(n) RETURN n will return all nodes, but MATCH p=()-[]-() RETURN preturns no nodes.

Hi @aayan636 thanks for reporting - we'll take a look at this at the earliest and get back with a fix.

Hi @aayan636, the issue should be more or less resolved as per https://github.com/run-llama/llama_index/pull/15889. I'm not sure how soon they'll merge this and when it'll be available on the LlamaHub package index for you to test this, but you can check out the LlamaIndex repo on your end and run your code by importing the local modules in the interim.
import sys
sys.path.append("/Users/prrao/code/llama_index/llama-index-integrations/graph_stores/llama-index-graph-stores-kuzu/llama_index/graph_stores/kuzu/")
from kuzu_property_graph import KuzuPropertyGraphStore

You might need to resolve the utils.py path as well to run this locally, as the structure of the code is designed to work as a package.
Hope this works!

Hi @prrao87, thank you so much for resolving this quickly. I'll check the fix locally on my end and will update the ticket once I have verified the patch works for me.

Merged 👍🏻 Should be published shortly

Great, thanks, Logan!


open-template doesn't work from within a component.js file

Is the project on which you tried this published?  I can't replicate failure here.

yes - for instance https://github.com/getslash/backslash - from within https://github.com/getslash/backslash/blob/master/webapp/app/components/query-result/component.js

Also tried setting usePods to true. Same issue.
On יום ה׳, 10 במרץ 2016 at 11:25 Aad Versteden<EMAIL_ADDRESS>wrote:

Is the project on which you tried this published? I can't replicate
failure here.
—
Reply to this email directly or view it on GitHub
https://github.com/madnificent/ember-mode/issues/12#issuecomment-194750073
.

it did not work initially, however when using the latest ember-mode with "usePods": true in /webapp/.ember-cli, it did yield the correct template file when calling M-x ember-open-template.
Do you have any errors in the *Messages* buffer? Do other movement commands work as expected?

I see these in messages:
checking filename app/templates/components/.gitkeep
checking filename app/templates/test-result.hbs
checking filename app/templates/single_error.hbs
checking filename app/templates/session-result.hbs
checking filename app/templates/errors.hbs
checking filename app/templates/error.hbs

Other movements work fine - it seems it's only broken for components

That is the output which I see here when PODs are not enabled.  You should be able to override POD detection by using the .dir-locals.el trick (check the README for that).  Are you setting "usePods" in the way I indicated above?  The detection is implemented using regexes, so nothing smart is allowed ;-)

Oh sorry - my bad. I accidentally reverted .ember-cli on one of my changes.
I still have a problem though - my repo uses a mix of pods and regular layout (all models are in models/ and the templates and controllers are in pods). I think it would be nice for ember-mode to fallback into the opposite option when not finding its target (like when looking for foo/model.js and not finding it, fallback to models/foo.js and the other way around. What do you think?


Endpoints setup for retrieving basic data from users, groups, activities tables

Endpoints listed below setup for retrieving (GETting) basic data from users, groups, and activities tables.
api/users
api/users/:id
api/users/:id/groupsBelongedTo
api/users/:id/groupsOwned
api/groups
api/groups/:id
api/groups/:id/groupMembers
api/groups/:id/groupOwners
api/groups/:id/activities
api/groups/:id/callStatus
api/activities
api/activities/:id

 Setup endpoints for retrieving basic data from users, groups, and activities tables.


[build-images] Updated to v31694071-c6e811e7

What does this PR do?
This updates the build images to have python-gitlab to btf-gen images

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


Report status via Checks API

It'd be cool to visualize the status and instructions on the Checks page.

Good idea - ref #115

More like https://github.com/adobe/cla-bot/pull/49/checks?check_run_id=84615061


[#328] Default Safe variable is incorrect in many cases

🗣 Description
Default Safe variable is incorrect in many cases
Closes #328
💭 Motivation and context
The Default Safe variable was assigned an incorrect value in many of the baselines and this was resulting in incorrect conditional checks when there are no log events present for a given policy.
🧪 Testing

Testing may involve testing in a default setting environment (potentially new environment) where the settings/controls are unchanged and there are no log events.

Another work around to test this is to delete the logs from the 'ProviderSettingExport.json' file, that way no log events will be present and the DefaultSafe value can be tested

This update touched many of the baseline policies and corresponding unit tests, so ensure everything is working as expected

✅ Pre-approval checklist

[ ] This PR has an informative and human-readable title.
[ ] Changes are limited to a single goal - eschew scope creep!
[ ] If applicable, All future TODOs are captured in issues, which are referenced in the PR description.
[ ] The relevant issues PR resolves are linked preferably via closing keywords.
[ ] All relevant type-of-change labels have been added.
[ ] I have read and agree to the CONTRIBUTING.md document.
[ ] These code changes follow cisagov code standards.
[ ] All relevant repo and/or project documentation has been updated to reflect the changes in this PR.
[ ] Tests have been added and/or modified to cover the changes in this PR.
[ ] All new and existing tests pass.

✅ Pre-merge Checklist

[ ] This PR has been smoke tested to ensure main is in a functional state when this PR is merged.
[ ] Squash all commits into one PR level commit using the Squash and merge button.

✅ Post-merge Checklist

[ ] Delete the branch to clean up.
[ ] Close issues resolved by this PR if the closing keywords did not activate.

@adhilto @mitchelbaker-cisa @LaurenBassett this PR is ready for review.


Need of Internet / dubious connections / spyware

I find it a bit strange that the internet is used all the time. Especially with so many IPs... Can someone explain why?
In the data I could find out that e.g. Google is used (I assume for their language model?)
Anyway, these are all the connections that are there: I find the ones with an “!” strange:

US:
9 x huggingface.co    ---> for what???
2 x httbin.org.   !    ---> what is that?
4 x api.gradio.app   ---> Lib?
1 x cdn-lfs-us-1.huggingface.co    ---> for what?

CN:
2 x ip.taobao.com   !    ---> Very suspicious ... Spyware? This is a online shop!

IE:
2 x checkip.amazonaws.com   !   ---> Why the HECK Amazon?

CA:
7 x geolocation.onetrust.com   !    ---> Something to detect which country?

DE:
2 x github.com    ---> for updating?

In addition, the following pages are also communicated when the browser is opened:

Cloudflare.com
fonts.googleapis.com
gstatic.com    ---> Why?

Don't take offense, but I'm being very careful here. In this project, various things are still installed after scripting... all in all, it seems too confusing for me to deal with. Perhaps the creator can shed some light on this?

First of all let me applaud you for your research here.
Next, I'll put some of your concerns at ease and raise others.
HuggingFace with LFS is probably used for the models and related things.
The GradioAPI is for Gradio telemetry data, collected by Gradio (for what purpose? most likely bug tracking of something, but who knows~).
httbin.org is generally used for testing, though a thorough dive into the code can uncover other uses :eyes:
checkip.amazonaws.com used by the Gradio telemetry api to get your IP.
geolocation.onetrust.com probably more telemetry, by whom will require some deep dive.
ip.taobao.com the Chinese AWS, what is hosted there will be interesting, also requires a deep dive into the code.
I'll check these out and comeback with insights.
It seem the OP is MIA for an entire quarter so probably we won't receive input from him.
But I'm curious, and before I sacrifice time for my own tool instead of this one I'll make sure it's safe.
Thanks again for bringing this to my attention.
Cheers~

Hey thx for the informations! :)
I found out some more info using Wireshark... But first a few comments and questions on what I know so far:
HuggingFace: But why do I need all the time a connection to huggingface, if I installed all modells 🤔
GradioAPI: Yeah, gradioapi ist strange... but no proof of misuse...
httbin.org: No answer but more information

httpbin is a popular online service that provides a simple HTTP request & response service for testing and debugging HTTP libraries and clients. Some key things you can do with httpbin
from proxiesapi.com

checkip.amazonaws.com: For what does it need my IP?
ip.taobao.com: I tracked the IP behind it. Seems to be a Service from Alibaba.com ... but no clue for what :/

New Information:
I used Wireshark to look at all http connections (really only http, as there is too much traffic over my machine). These companies are always retrieved at the start of finetune or xtts webui (I'm in Germany, therefore Akamai in DE)
Akamai Technologies Inc.	DE --> "OCSP 544 Request"
EU Metro Frontend 		IR --> "OCSP 540 Request"
Edgecast Inc			US (California) 3 times reported, but no evidence of malware... just a OCSP 544 Request
Amazon.com Inc.		US (Virginia) --> Probably IP as mentioned by AnonimusJack "OCSP 533 Request"
Google LLC			US (Missouri) --> "403 GET /success.txt?ipv4 HTTP/1.1" and "423 GET /success.txt?ipv6 HTTP/1.1"🤷

What I find even more interesting is that during the process of cloning a voice from my own model, I occasionally get a Peer2Peer connection to a IP in China:
<IP_ADDRESS>              China Telecom			Beijing
!!! I have no proof that this IP is directly related to the XTTS WebUI or TTS !!!!
But perhaps someone has observed similar behavior?

I found something more.
It seems most of the "problematic" API calls come from this library: https://github.com/uliontse/translators
python class Region(Tse):     def __init__(self):         super().__init__()         self.get_addr_url = 'https://geolocation.onetrust.com/cookieconsentpub/v1/geo/location'         self.get_ip_url = 'https://httpbin.org/ip'         self.ip_api_addr_url = 'http://ip-api.com/json'  # must http.         self.ip_tb_add_url = 'https://ip.taobao.com/outGetIpInfo'         self.default_region = os.environ.get('translators_default_region', None)
From translators/server.py line 322.
Seems kinda safe.
I should check the project, maybe it can be turned off if there's no need to detect a language or do some translation for some peace of mind :grin:


How to make flutter (and dart) more popular?

By officially supporting this https://github.com/flutter2js/flutter2js
People will sold if one code can support all 3 most popular platform (web, android, iOS)

This shouldn't be a place for advertising.

why is this a issue?, the flutter and dart popularity are already increasing.

This ticket should be closed. There are currently 3,331 open Issues, we should get that number down.

This was asked a lot, but it's not too easy.
It was also explored in https://github.com/yjbanov/butterfly.
AFAIK there are currently no such plans in that direction.


Consider not building the docs for the nightlies

This would greatly reduce build time, and the size of the nightlies; but I need to consider the impact on users.
So I need to ask the question: how useful/vital are the installed docs when developing on ARM devices?
When developing in amd64/arm I usually use the hosted documentation (or just browse through the source using my editor). So I'd be in favor of removing the docs from the nightlies.
I'll like to hear other people's opinion.
(Probably it's too early to ask, since it's been only a few days since I've started shipping the final form (RPi compatible) of the nightlies, so take your time to answer!)
cc @Novacha @shavounet
For the record building + uploading the nightlies takes:

with docs: 3h 21m 59s
without docs: coming soon

I use the online docs right now even though rustup installs them to my computer, so I wouldn't have a need rust docs to be on the ARM device. I also assume that the documentation would be exactly the same as the documentation generated by the rust linux buildbot and uploaded to the rust doc site, so it would superfluous to build them again.
I'd say that it's probably not worth the extra build time and size.

Thanks for the input @Novacha! And yes, the docs should match since the nightlies are built from the same commit hash.
As an experiment, I'm building the next rust nightly without docs to get an idea of the reduction of build time/tarball size.

On Gentoo, ebuilds have a doc USE-flag for this, optionally allowing you to install the documentation in conjunction with the software. However, most users turn this off on a system-wide level, to save disk space (lots of it, imagine having this enabled for every common package, and having a beast like texlive installed) and because all the documentation is usually online somewhere. I am one of those users, as my Chromebooks only have 16 GiB or 32 GiB of space in total. I do have some ARM devices with a 120 GiB SSD attached to them via SATA, but I don't use those often, and still wouldn't populate it with documentation. Nevertheless, if I really wanted to, I could just enable the USE-flag and build it myself, which means there is no need for me to have documentation preshipped with the binaries.
To me offline documentation is only useful in situations where you have no internet connection, or a poor one. However, if there are people who want it, and it takes a lot of time to build, then it might be worth it to build it every ten releases or so.

Bah, --disable-docs only shaves 12 mins (6%) off the build time, and reduces the tarball size by 33 MB (25%).
I thought the win in build time was way bigger than this. At this point, I don't mind building the docs for every nightly. I guess the question is now whether you mind the extra bandwidth required to download a nightly with docs?

I'll continue to bundle the docs with the nightlies to be consistent with the official nightlies.
Thanks everyone for the input!


Test-SecretVault does not take wildcards for -Name parameter

It is not easy to test all the secret vaults as the name parameter does not take wildcards. Get-SecretVault does take wildcards for name which set the expectation that Test-SecretVault will accept it too.
Current state:
PS /> Get-SecretVault
Name         ModuleName                       IsDefaultVault

SecretStore  Microsoft.PowerShell.SecretStore False
SecretStore2 Microsoft.PowerShell.SecretStore True
Actual
PS /> Test-SecretVault -name *
Test-SecretVault: Vault not found in registry: *
Expected
Should test all registered secret vaults.
Workaround
Get-SecretVault -Name * | Test-SecretVault

It seems this issue should be in SecretManagement repo, can this be moved please.


upgrade to support solidity >= 0.5.0, web3 >= 1.0.0, truffle >= 5.0.0

dead repo and I am not on the team, but LGTM hahahaha


Can this plugin be used with Docker containers? If Yes, please point to the documentation.

What issue would you like to bring attention to?
What is the impact of this issue?
Do you have a proposed response or remediation for the issue?

Hi @sumitk31 I never try something like that so I don't know. This CNI plugin is to be used as a secondary nic in k8s by multus-cni

closing this issue feel free to reopen if needed


Changing WLAN Access point

When I change my WLAN access point while Synctrayzor was already running, I get this message:
Syncthing is deactivated
Do you want to change the operating conditions?
Reasons:
Connected with no WLAN

When I reboot the device (Xiaomi Poco X3Pro), it works.
Ist there a way to prevent rebooting every time the phone switches the accesspoint?

As far as I know, SyncTrayzor only is a desktop app for Windows. You are reporting an issue with the mobile app of SyncThing, unrelated to this SyncTrayzor app.
As a user of SyncThing android app, I can suggest you check the settings of the app : Settings > Run Conditions
These "run conditions" allow you to customize in which cases SyncThing will run (wifi, metered wifi, mobile data, etc.), or be disabled.


prefetch_factor

fastNLP\core\dataloaders\torch_dataloader\fdl.py in init(self, dataset, batch_size, shuffle, sampler, batch_sampler, num_workers, collate_fn, pin_memory, drop_last, timeout, worker_init_fn, multiprocessing_context, generator, prefetch_factor, persistent_workers, **kwargs)
150                              persistent_workers=persistent_workers)
...
--> 245             raise ValueError('prefetch_factor option could only be specified in multiprocessing.'
246                              'let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.')
247         elif num_workers > 0 and prefetch_factor is None:
ValueError: prefetch_factor option could only be specified in multiprocessing.let num_workers > 0 to enable multiprocessing, otherwise set prefetch_factor to None.

have same problem, set prefetch_factor param value to None can fix this error
fastNLP==1.0.1
fasthan==2.0
path:
FastModel.py#__preprocess_sentence
dataloader = TorchDataLoader(dataset, **prefetch_factor=None,**
                                     batch_sampler=BucketedBatchSampler(
                                         dataset=dataset,
                                         batch_size=1,
                                         length='seq_len'))


Add Tkinter

Tool information

Tool name: Tkinter
Add or update? Add, ish
Desired version: latest
Approximate size:
If this is an add request:

Brief description of tool: Enables the use (or at least import of) the tkinter Python module in the standard library
URL for tool's homepage: https://wiki.python.org/moin/TkInter

Area for Triage:
Python
Packages
Question, Bug, or Feature?:
Feature, I guess?
Virtual environments affected

[x] macOS 10.15
[x] Ubuntu 16.04 LTS
[x] Ubuntu 18.04 LTS
[x] Windows Server 2016 R2
[x] Windows Server 2019

Can this tool be installed during the build?
Not to my knowledge. Attempting to install python3-tk and tk-dev before @actions/setup-python, but it doesn't seem to work. Likely requires a rebuild of Python.
Are you willing to submit a PR?
If I knew what to do, sure.

Tkniter already is present in Windows and MacOS images, but because it's a GUI interface, you can't check it from serial console, where you'll lost control, when you run tkniter application.
Additional libraries have been added to Ubuntu images for Tkniter support

We have tested all Ubuntu version and have got the correct results from test python script - closing the ticket.
Please feel free to reopen this ticket, if you'll find some issues.

To be fair, the Windows part of the issue wasn’t really addressed in a descriptive way about how to get around it as opposed to skipping tests or whatever regarding TKinter and Windows.

Hello @miketrethewey, thank you for your response. We use PyInstaller to install Python versions on  Windows image. It includes Tkinter by default so everything should work.
Example for Python 3.7:
import sys
print(sys.version)
import tkinter as tk
print(tk.TkVersion)

Output:

If it doesn’t work for you, could you please share minimal repro steps?

Needed a series of hacks to make it work. Our tests include making a Tk() element and it's not happy because of that.
Here's an Actions Run that did work properly.

@dmitry-shibanov Please see above. Is there a way to streamline that or make it more-visible since people still seem to be running into it?

Hello, @miketrethewey, thank you for your response. Am I right that you mention the following code?:
        disp=:99
        screen=0
        geom=640x480x24
        exec Xvfb $disp -screen $screen $geom 2>/tmp/Xvfb.log &
        export DISPLAY=:99
        python tkinter_issue.py

In our understanding this configuration is user specific and we shouldn’t set it in image generation.


Feature/login

:warning: You need to have the auth microservice up to date to test this PR :warning:
This pull request introduces several new components and features, updates dependencies, and makes various improvements to the codebase. The most important changes include adding a login page with form handling, creating reusable UI components, and updating dependencies in package.json.
New Components and Features:

Added a Login page with a LoginForm component that includes form validation and submission handling using react-hook-form and zod. [1] [2] [3]
Created a PasswordInput component that allows users to toggle password visibility.
Implemented a PulsingIcons component that displays animated icons.
Added new UI components including Card, Form, and Label with various subcomponents for building forms and cards. [1] [2] [3]

Dependency Updates:

Updated package.json to include new dependencies: @hookform/resolvers, react-hook-form, and zod. [1] [2]

Code Improvements:

Updated ModeToggle component to improve the transition between light and dark modes.
Added animations for pulsing and error states in animations.css.

These changes enhance the application's functionality, improve user experience, and ensure better code organization and maintainability.

Shouldn't we have an issue for this task?

I was getting a CORS error when trying to log in:

To make it work, I had to add the following CORS configuration to api.js file in the authorization microservice:
app.use(cors({ origin: process.env.CORS_ORIGIN, credentials: true }));

I was getting a CORS error when trying to log in:
To make it work, I had to add the following CORS configuration to api.js file in the authorization microservice: app.use(cors({ origin: process.env.CORS_ORIGIN, credentials: true }));

This is not enough to make it work when the gateway is introduced and the host are differents. The fix for this is on the api-gateway


How to refresh Paragraph

How to refresh Paragraph pls help :)

Hi, you can create a function to refresh your paragraph.
local Paragraph = Tab:AddParagraph({
    Title = "Title Here",
    Content = "Test 1"
})

Tab:AddButton({
    Title = "Refresh Paragraph",
    Callback = function()
        Paragraph:SetDesc("Test 1") -- if it's a table you can put :SetDesc(table.concat(<table>, <seperator>))
    end
})


integer support, transform debugging and more consistent bounds propagation

integer support (specifically in resize, clamp)
continuous bounds propagation through add, sub, mul, div
categorical bounds propagation through all binary ops
consolidated unary bounds propagation
materialize robustness to prevent privacy errors
optimize away clones in materialize
catch a few missed properties when clamping
temporarily disable Stack, Partition and ExponentialMechanism components
clean dead arguments out of JSONs
temporarily disable float categoricals
link to proofs from documentation (albeit somewhat hidden)
support left broadcasting and transforms on matrices, not just vectors
initial set of overflow/underflow bounds checking
enforce resize n > 0 (technically it could be n >= 0 and still be private)
initial set of deduplication/non-duplicated categories checks

This is ready for review.


Gedmo Timestampable triggers prePersist events when using findBy

Describe the bug
Whenever an entity has a Gedmo Timestampable field, if a simple findBy (SELECT) is made on this entity, Gedmo will trigger the prePersist event, even if there's no insert / update made on this entity.
To Reproduce

Create an Entity and add a createdAt + updatedAt field with the associated PHP Attributes for the Gedmo Timestampable
Create an EventListener with the prePersist event listened
Use a simple findBy on the Entity's Repository

Expected behavior
The EventListener is not triggered by the findBy method.
Actual behaivour
The EventListener is triggered by the findBy method.
Additional context
Tested with Symfony 6.4 and PHP 8.3 on the latest version of this bundle + latest version of gedmo.

Please report that to the extensions themselves. This bunde is not the repository where the logic is implemented.


test/number prototype chr

test: initial test set up
test: provides tests for typescript Number chr method
Resolves issue #113

I can see the file at https://github.com/code-chronicles-code/leetcode-curriculum/blob/2ad7586de4ae68d10661131c29882dea5d5cf920/workspaces/adventure-pack/goodies/typescript/Number.prototype.chr/index.test.ts but yeah, for some reason GitHub thinks it's a binary file. @Gobleizer I'm not sure which of the non-ASCII characters is the culprit... Emojis are usually fine.

I believe it is the Null Character (https://unicode-explorer.com/c/0000) which is the result when you attempt to get a character from number 0.
I can see it won't appear in the file preview. But it was necessary to make the test pass, because a null character is not the same as an empty string.

Perhaps we should use escapes for the null character? So "\0", similar to "\n" for newlines?

Awesome point. Fixed.

What do you all think if I added inner describe blocks?
I like having it.each, but I dislike not having clear titles as to what a group of tests is doing right at the top of that group in code.
The test print out is actually better now since I can print unique strings for each test, which is fun.

Yeah I went ahead and nested the describe blocks.

Hey @Gobleizer I took this PR as-is since I'm about to work on some changes that might otherwise cause you some merge conflicts. Feel free to do a follow-up with any additional changes from the discussion.
Thanks again for working on this!


Revise Docker publish instructions to run commands in background

Fixes: #n/a

Docker build/push completes much more quickly by running commands in parallel
Also remove the "test the images locally" as that is superseded by our actual integration test suite

@jathanism doesn't look like this is relevant with the recent CI changes. Safe to close?


Missing method "SKPath.AddPath(SKPath childPath)"

SKPath is missing the method "AddPath(SKPath childPath)"
see:
void SkPath::addPath(const SkPath& path, SkScalar dx, SkScalar dy, AddPathMode mode)
in https://github.com/google/skia/blob/master/src/core/SkPath.cpp
I need this method, as I am replacing Android.Canvas with SkiaSharp
and Android.Graphics.Path has such an "AddPath" method
https://developer.android.com/reference/android/graphics/Path.html
I am using 1.49.4-beta

I have added the C API for Skia, =but have not done the binding yet.

Implemneted